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This  guidebook  is  one  of  a  series  of  guidebooks  Intended  to  assist 
Air  Force  Progran  Office  and  Engineering  personnel  in  software  acquisition 
engineering  for  airborne  systems.  The  contents  of  the  guidebooks  will  be 
revised  periodically  to  reflect  changes  in  software  acquisition  policies 
and  practices  and  feedback  from  users. 

This  guidebook  was  prepared  under  the  direction  of  the  Aeronautical 
Systems  Division,  Deputy  for  Engineering  (ASD/EN)  in  coordination  with 
the  Space  Division,  Deputy  for  Acquisition  Management  (SD/AQM). 

The  entire  series  of  Software  Acquisition  Engineering  Guidebooks 
(Airborne  Systems)  is  listed  below  along  with  ASD  Technical  Report  numbers 
and  NTIS  accession  numbers  where  available. 
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1.  INTRODUCTION 


1.1  PURPOSE  AND  SCOPE 

The  purpose  of  this  guidebook  is  to  identify  documentations,  methods, 
and  procedures  for  use  in  planning,  monitoring,  controlling,  and  reporting 
software  development. 

Extensive  documentation  exists  on  software  development  planning 
and  status;  however,  it  is  the  goal  of  this  guidebook  to  reduce  volumes  of 
data  to  a  more  concise,  useful  format  in  the  context  of  avionics  and  space- 
borne  software  development  efforts.  The  procedures  and  methods 
described  herein  help  to  identify  the  types  of  information  pertinent  to 
a  project  and  how  to  use  them  in  emphasizing  initial  planning  and  deter¬ 
mining  status  of  the  software  development, 

1.2  RELATIONSHIPS 

1.2.1  Life  Cycle  Relationships 

Software  development  planning  begins  with  initial  focus  on  producing 
a  system  involving  computer  control  or  utilization  and  continues  until  the 
system  is  completely  developed.  This  guidebook  primarily  applies  to 
those  computer  programs  developed  within  the  normal  weapon  system  life 
cycle  sis  shown  in  Figure  1-1.  However,  certain  software  acquisitions 
may  occur  within  only  one  life  cycle  phase  (of  the  system)  while  others 
may  span  several  phases.  Monitoring  and  control  procedures  must  be 
adjusted  accordingly. 

1.2.2  Relationship  to  Other  Guidebooks 

The  degree  of  adherence  to  methods  and  procedures  outlined  in  the 
other  guidebooks  directly  correlates  to  the  software  development  progress. 
Proper  configuration  management  or  proper  quality  control  have  intangible 
payoffs  that  obviously  affect  development  progress.  Absence  of  such 
disciplines  causes  a  severely  detrimental  effect  on  progress  yet,  to  the 
uninformed,  enforcement  of  these  disciplines  seems  to  yield  little  except 
increased  costs.  This  guidebook  supplements  those  emphases  made  in 
other  series  guidebooks. 
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SYSTEM  LIFE-CYCLE  PHASES 


Provides  background  information  on  the  use  of  software  development 
planning  and  status  as  they  apply  to  the  system  life  cycle  and  to  other 
guidebooks  published  in  this  series. 

1.3.2  Section  2;  Relevant  Documents 

*  References  the  government  regulations  and  standards  which  relate 

\  to  software  development  planning  and  status  reporting. 

1.3.3  Section  3:  General  Guidelines  for  Software  Development 
Planning  and  Control 

Provides  general  guidance  to  the  Program  Office  engineers  and 
managers  in  the  planning  for  software  development.  It  further  addresses 
monitoring,  controlling,  and  reporting  software  development  progress, 
techniques,  and  methodology  currently  in  use,  factors  which  affect  de¬ 
velopment  progress  and  associated  risks. 

1.3.4  Section  4;  Implementing,  Monitoring,  and  Reporting 
Development  Status 

Provides  guidance  on  software  development  monitoring,  controlling, 
and  reporting.  It  provides  discussion  on  milestones  and  schedules,  re¬ 
views,  and  development  status  reporting. 

1.3.5  Appendices  A  Through  E 

Provides  additional  information  on  planning  documents  and  the  eval¬ 
uation  of  them. 

1.3.6  Appendix  F.  Annotated  Bibliography 

Lists  a  number  of  references  that  provide  information  on  various 
subjects  related  to  software  development,  status,  and  test. 
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2.  RELEVANT  DOCUMENTS 

The  following  documents  are  important  sources  of  information  rele¬ 
vant  to  software  development  planning  and  control.  Additional  sources  of 
background  information  related  to  software  development  and  control  are 
provided  in  Appendix  D . 


DODD  4120.3 

Defense  Standardization  Program 

DODD  5000.1 

Major  System  Acquisition 

DODD  5000.2 

Major  System  Acquisition  Process 

DODD  5000.11 

Data  Elements  and  Data  Codes  Standardi¬ 
zation  Procedures 

DODD  5000.29 

Management  of  Computer  Resources  in 

Major  Defense  Systems 

AFM  26-1 

Manpower  Policies  and  Procedures 

AFR  74-1 

Quality  Assurance 

AFR  80-14 

Test  and  Evaluation 

AFR  300-10 

Computer  Programming  Languages 

AFR  310-1 

Management  of  Contractor  Data 

AFR  800-2 

Program  Management 

AFR  800-3 

Engineering  for  Defense  Systems 

AFR  800-8 

Integrated  Logistics  Support  (ILS)  Program 
for  Systems  and  Equipment 

AFR  800-11 

Lite  Cycle  Cost  Management  Program 

AFR  800-12 

Acquisition  of  Support  Equipment 

AFR  800-14 

Management  of  Computer  Resources  in 

Major  Defense  Systems  (Volumes  I  and  II) 

AFR  800-28 

Air  Force  Policy  in  Avionics  Acquisition  and 
Support 

AFSCM  375-7 

Configuration  Management  for  Computer 
Programs 

AFSCR  800-1 

Command  Review  of  Systems  Acquisition 
Programs  and  Test  Resources 

I 
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DAR  1-2102  Procurement  Planning 

MIL -STD  483  Configuration  Management 

MIL-STD  490  Specification  Practices 

MIL-STD  499A  Engineering  Management 

MIL-STD  1521A  Technical  Reviews  and  Audits  for  Systems, 

Equipment,  and  Computer  Programs 

MIL-S  52779A  Quality  Assurance 

SAMSO  STANDARD  Standard  Engineering  Practices  for  Computer 

73-3  Software  Design  and  Development 


DAR  1-2102  Procurement  Planning 

MIL -STD  483  Configuration  Management 

MIL-STD  490  Specification  Practices 

MIL -STD  499A  Engineering  Management 

MIL-STD  1521A  Technical  Reviews  and  Audits  for  Systems, 

Equipment,  and  Computer  Programs 

MIL-S  5277 9A  Quality  Assurance 

SAMSO  STANDARD  Standard  Engineering  Practices  for  Computer 
73-3  Software  Design  and  Development 
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3.  GENERAL  GUIDELINES  FOR  SOFTWARE 
DEVELOPMENT  PLANNING  AND  CONTROL 

Air  Force  Regulation  800-2  requires  each  acquisition  program  to  be 
managed  by  a  single  person  known  as  the  Program  Manager  (PM)*  This 
guidebook  provides  information  to  the  PM  or  his  representatives  for 
planning  and  controlling  software  development  pertinent  to  his  acquisition. 
Control  of  software  development  is  possible  only  to  a  level  equivalent  to 
the  planning  level  associated  with  the  software  development.  Top  level 
project  planning  will  enable  only  top  level  project  control  even  though 
detailed  control  procedures  (e.g.,  software  control)  are  attempted.  Since 
successful  development  control  is  highly  dependent  on  adequate  develop¬ 
ment  planning,  this  section  emphasizes  the  planning  documents  which 
facilitate  software  development  allowing  subsequent  development  control. 
Table  3-1  shows  the  relationship  between  various  planning  documents 
discussed  in  this  section. 

3.1  SOFTWARE  ACQUISITION  MANAGEMENT  PLANS 

Planning  is  a  prerequisite  to  efficiently  accomplishing  any  sophis¬ 
ticated  task.  Since  it  is  nearly  impossible  to  devise  a  plan  with  100% 
accuracy  and  efficiency,  initial  planning  will  only  approximate  the  actual 
situation.  Experienced  managers  lay  out  a  planned  course  of  action  based 
upon  current  data  and,  as  the  data  changes,  change  the  plan  accordingly; 
however,  a  change  is  not  adopted  unless  some  improved11  data  are  per¬ 
ceived.  This  section  assists  in  initially  laying  out  a  software  project 
within  a  larger  System  Acquisition  Program.  The  topics  discussed  rep¬ 
resent  coverage  of  the  prime  concerns  in  initial  planning  yet,  by  no 
means,  cover  all  planning  aspects. 

3.1.1  Procurement  Plan 

One  of  the  major  initial  tasks  in  any  program  is  to  develop  an  acqui¬ 
sition  strategy  for  the  total  system.  Technical,  business,  and  manage¬ 
ment  areas  are  addressed  to  provide  a  basis  for  integration  of  these  areas 
to  achieve  program  objectives.  The  strategy  is  expanded  and  refined  as 
the  program  progresses.  Schedules  and  funding  plans  are  p?*j>ared  to  ac¬ 
comodate  areas  of  uncertainty  and  risk.  Usually,  the  system  requirements 
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Table  3-1.  Software  Development  Planning  Documents 


Document 

Usual 

Ty] 

pe 

Requirement 

Documents 

Developer 

Mgmt 

Tech 

Program  Management  Plan 

Program 

Office 

X 

AFR  800-2 

Procurement  Plan 

Program 

Office 

X 

DAR  1-2102 

Test  and  Evaluation  Master 

Plan 

Program 

Office 

X 

AFR  80-14 

Computer  Program  Development 
Plan 

Contractor 

X 

X 

AFR  800-14 

Configuration  Management  Plan 

Contractor 

X 

MIL-STD  483 
Appendix  I 

Quality  Assurance  Plan 

Contractor 

X 

AFR  74-1  and 
MIL-S  5277  9A 

Computer  Resources  Integrated 
Support  Plan 

CRWG 

X 

X 

AFR  800-14 

Operational /Support  Configura¬ 
tion  Management  Procedures 


User/ 

Supporter 


X 


AFR  800-14 


are  defined  only  in  operational  terms,  software  requirements  are  vaguely 
stated,  and  management  control  is  not  yet  formalized.  However,  for 
budgeting  purposes,  in-depth  estimates  to  the  DOD  funding  cycle  are  neces 
sary.  Although  project  uncertainty  exists,  the  Procurement  Plan  (PP) 
outlines  an  initial  course  of  action  which  will  influence  the  entire  system 
development. 

3.1.1.1  Software  Considerations  in  the  Procurement  Plan 

The  basic  objective  of  the  Procurement  Plan  is  to  facilitate  attain¬ 
ment  of  the  procurement  objectives.  A  PP  is  prepared  concurrently  with 
the  request  for  program  funding  and  provides  a  matrix  for  the  integration 
and  coordination  of  personnel  engaged  in  the  determination  of  require¬ 
ments,  development  of  a  technical  data  package,  funding,  contracting, 
and  contract  administration.  Software  is  usually  intertwined  into  all  of 
these  activities  and  can  provide  a  substantial  influence  on  developments 
costs.  At  a  minimum,  software  related  questions  such  as  the  following 
should  be  addressed 

i.  Will  the  software  be  a  separate  procurement  from  the  rest 
of  the  system? 

2 •  Will  the  software  effort  involve  a  source  selection  process? 

3.  How  will  selection  criteria  be  established? 

4.  Who  will  govern  the  source  selection  and  publish  its  procedures 
and  proceedings? 

5.  Will  the  software  be  accomplished  by  multiple  contractors? 

6.  Will  the  procurement  involve  independent  verification  and 
validation? 

7.  How  will  the  lead  contractor  be  established? 

8.  Will  the  Air  Force  organically  develop  any  portions  of  the 
software? 

9.  What  type  of  contract(s)  is  required? 

10.  Are  data  rights  expected  to  be  a  factor? 

11.  What  technical  and/or  schedule  risks  are  evident? 

Appendix  A  contains  a  list  of  information  required  by  DAR  1-2102  to 
be  included  in  the  Procurement  Plan. 


3. 1.1.2  Software  Developmerit/Visibility  Through  Use  of  Procurement 
Plan 

Direct  control  of  software  development  is  somewhat  ineffective  when 
using  only  those  items  contained  within  the  PP;  however,  at  this  point, 
the  acquisition  method  of  developing  software  has  already  been  determined. 

Technical  and  schedule  risks  have  been  estimated  or  flagged  as  problem 
areas  so  management  adjustments  can  be  made  accordingly.  In  summary, 
the  PP  has  defined  the  acquisition  strategy  at  a  top  level  so  control  can  be 
exerted  at  a  commensurate  level. 

J 

3.1.2  Program  Management  Plan 

Air  Force  Regulation  800-2  requires  the  Program  Manager  to  pre¬ 
pare  a  Program  Management  Plan  (PMP)  tailored  to  the  needs  of  the  pro¬ 
gram.  The  PMP  is  a  directive  to  all  participating  commands  and, 
although  the  direct  responsibility  of  the  Program  Manager,  is  jointly  pre¬ 
pared  and  coordinated  with  these  commands.  Updates  to  the  PMP  should 
be  made  whenever  significant  program  changes  occur  or  the  data  contained 
are  obsolete. 

3. 1.2.1  Software  Considerations  of  the  Program  Management  Plan 

The  PMP  is  developed  and  issued  by  the  Program  Manager  to  show 
how  the  tasks  described  in  the  Program  Management  Directive  (PMD) 
will  be  accomplished.  It  includes  complete  planning  for  the  acquisition 
management  of  the  computer  resources.  Specifically,  AFR  800-14 
Volume  II  requires  coverage  of  the  following,  when  applicable: 

a.  Operational  and  support  concepts  for  computer  programs. 

b.  Identification  of  technical  and  managerial  expertise  allocated 
to  the  program  office. 

c.  Systems  Engineering  approach  to  be  followed. 

d.  Coverage  of  appropriate  hardware  tradeoffs. 

e.  Standardization  and  commonality  considerations. 

f.  Requirements  for  computer  program  and  data  rights  consistent 
with  the  planned  operational  and  support  concepts. 

g.  Master  schedule  of  major  milestones. 
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Identification  of  required  interfaces  between  this  system 
and  other  systems. 

i.  Requirements  for  growth  capability. 

j.  Requirements  for  acquisition  and  support  of  documentation. 

k.  Requirements  for  simulation,  integration,  and  other  facilities 
necessary  to  support  computer  programs. 

l.  Configuration  management  concepts. 

m.  Program  Management  Responsibility  Transfer  criteria. 

n.  Preparation  of  a  Computer  Resources  Integrated  Support  Plan 
(CRISP). 

Once  established,  the  PMP  serves  as  the  basic  overall  planning 
document  for  guiding  management  and  technical  exchanges  below  the 
Air  Staff  level.  Command  responsibilities  are  established  in  conjunction 
with  projected  milestones  enabling  intra -Command  planning  to  proceed 
at  the  next  lower  level.  The  PMP  should  be  kept  current  through  regular 
updates,  or  if  necessary,  to  reflect  actual  program  changes. 

The  following  sequence  of  steps  generally  describes  the  generation 
of  a  PMP  for  a  system,  and  in  addition,  points  out  some  of  the  software 
unique  concerns. 

•  Step  1  -  Identify  a  single  focal  point.  Whether  the  Program 

Manager  personally  spearheads  the  PMP  development  or  dele¬ 
gates  this  responsibility,  it  is  important  that  a  single  focal 
point  be  designated  for  continuity  and  coordination  purposes. 

•  Step  2  -  Identify  any  constraints.  A  compressed  schedule, 

inadequate  funds,  use  of  a  particular  contractor,  or  an  intense 
operational  need  are  the  kinds  of  constraints  which  drive 
programs.  These  can  lead  to  eliminating  and/or  overlapping 
phases  in  the  system  life  cycle.  Constant  management  atten¬ 
tion  is  necessary  to  counterbalance  any  of  these  constraints. 

•  Step  3  -  Scope  the  project.  It  is  necessary  to  scope  any  task 

(especially  software)  to  adequately  plan  its  development.  A 
reasonable  margin  for  error  should  be  allowed  in  the  scoping 
and  can  be  included  in  both  cost  and  schedule  estimates.  Con- 
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sidering  any  identified  constraints,  provide  a  generalized 
estimate  of  the  job  broken  out  by  desired  category  (hardware, 
software.  Computer  Program  Configuration  Item  (CPCI),  or 
milestone).  Use  experience  factors  gained  from  previous 
successful  programs  of  comparable  complexity  realizing  that 
cost  and/or  schedule  revisions  will  be  likely  as  improved 
data  are  received.  Consider  interfaces  with  other  systems  and 
the  relative  stability  of  those  systems.  The  results  of  this 
step  should  broadly  scope  the  task,  by  category,  and  will  serve 
as  the  initial  planning  estimates  for  the  PMP.  During  the  early 
phases  of  avionics  projects,  scoping  of  the  subsystems  may 
be  a  combined  hardware/software  estimate.  Detailed  con¬ 
tractor  trades  will  ultimately  define  the  hardware /software 
dividing  line. 

•  Step  4  -  Conceive  an  overall  project  approach.  Operational 
needs  have  provided  the  basis  for  system  requirements;  these 
requirements  should  be  used  to  specify  a  sequence  of  steps 
by  which  the  system  will  be  developed,  for  example: 

a.  Hardware  and  software  to  be  developed  in  parallel 
by  one  contractor. 

b.  System  will  consist  of  four  CPCI's. 

c.  Joint  test  program  using  AFTEC,  user,  and  support¬ 
ing  commands. 

d.  Organic  support  during  operational  use. 

•  Step  5  -  Develop  a  Master  Schedule.  Combining  Steps  3  and 
4  should  provide  a  milestone  schedule.  If  risks  are  substan¬ 
tial,  then  factor  these  in  as  management  judgments. 

•  Step  6  -  Identify  resources.  Standard  personnel  and  equip¬ 
ment  productivity  factors  applied  to  Step  5  should  yield  an 
indication  of  the  recources  necessary  to  accomplish  the  devel¬ 
opment.  Forecast  the  source  and  date  of  procuring  these 
resources.  Identify  expected  tasks  and  schedules  for  the 
participating  commands. 


-  12  - 


1 


3. 1.2. 2  Development  Visibility /Control  Through  Use  of  the  Program 
Management  Plan* 

The  Program  Management  Plan  primarily  provides  management 
control  of  the  system  development  including  software.  Top  level  approaches 
to  system  engineering,  schedule  control,  configuration  management  con¬ 
cept.  and  methods  of  engineering  and  status  data  acquisition  are  defined 
in  the  PMP.  The  PMP  further  outlines  the  areas  of  responsibility  for 
all  Air  Force  and  contractor  participants.  Additionally,  the  document 
outlines  the  method  by  which  system  and/or  software  tradeoffs  may  be 
accomplished.  The  PMP  schedules  establish  estimated  dates  for  which 
the  control  documents  such  as  the  Computer  Program  Development  Plan 
(CPDP)  and  the  Configuration  Management  Plan  (CMP)  will  be  produced. 

In  summary,  completion  of  the  PMP  allows  the  orderly  commencement 
of  the  development  effort,  and  implementation  of  the  PMP  allows  continued 
orderly  development. 

3.1.3  Test  and  Evaluation  Master  Plan 

As  early  as  possible,  management  should  conceptualize  and  scope 
the  testing  to  be  accomplished.  The  test  section  of  the  Program  Manage¬ 
ment  Plan  contains  the  generalized  test  objectives  to  be  fulfilled;  however, 
an  overall  test  and  evaluation  plan  to  identify  and  integrate  the  test  effort 
and  schedules  is  also  needed.  AFR  80-14  defines  such  a  plan  and  labels 
it  the  Test  and  Evaluation  Master  Plan  (TEMP).  The  TEMP  should  be 
generated  prior  to  full-scale  engineering  development  and  coordinated 
with  AFTEC  and  other  major  commands  as  appropriate.  All  changes  to 
the  TEMP  should  require  formal  approval  and  should  be  accompanied  by 
rationale  for  the  changes.  The  document  must  be  kept  current  for  all  key 
test  and  evaluation  decision  points. 

3. 1.3.1  Software  Considerations  of  the  Test  and  Evaluation  Master  Plan 

The  TEMP  will  identify  all  system  testing  to  be  accomplished.  Soft¬ 
ware  testing  (such  as  exercises,  simulations,  etc. )  must  be  conceived 
and  integrated  into  the  overall  plan.  Each  test  effort  and  its  associated 
schedule  must  be  listed.  All  test  support  agencies  must  be  contacted 
with  a  statement  of  expected  test  participation  and  support.  The  type  of 
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testa  and  what  is  to  be  tested  must  be  identified.  A  description  of  manage¬ 
ment  and  control  procedures  related  to  testing  and  evaluation  must  be 
provided.  Figure  3-2  represents  a  hierarchical  flow  for  the  various  test 
documents. 

3.1. 3. 2  Development  Visibility/Control  Through  Use  of  the  Test  and 
Evaluation  Master  Plan 

The  Test  and  Evaluation  Master  Plan  (TEMP)  should  insure  that 
management  responsibilities  and  control  mechanisms  have  been  outlined 
for  any  necessary  software  testing  as  well  as  system  testing.  This  doc¬ 
ument  will  indicate  any  test  schedule  or  integration  conflicts  and  thus 
the  conflicts  may  be  resolved  early  in  the  development  cycle.  Early 
indications  of  problem  areas,  whether  of  technical  origin  or  schedule/ 
integration  origin,  serve  to  minimize  development  costs.  The  TEMP 
outlines  the  overall  approach  to  testing  and  thus  provides  a  baseline 
against  which  project  management  may  control  the  test  portion  of  soft¬ 
ware  development. 

3.1.4  Computer  Resources  Integrated  Support  Plan 

Section  3.3.1  of  this  guidebook  specifically  addresses  the  Computer 
Resources  Integrated  Support  Plan  (CRISP)  document  and  its  preparation. 
This  section  focuses  on  the  usefulness  of  the  CRISP  to  early  project 
planning. 

Assurance  of  adequate  support  of  computer  resources  after  manage¬ 
ment  responsibility  has  transferred  to  the  supporting  command  is  the  main 
purpose  of  the  CRISP.  Early  project  planning  must  include  provision 
for  acquiring  adequate  documents,  computer  equipment,  and  computer 
programs  necessary  to  provide  support.  Definition  and  possible  pro¬ 
curement  of  these  equipments,  facilities,  and  software  are  the  responsi¬ 
bility  of  the  Program  Office  (to  the  extent  specified  in  the  PMD)  and  must 
be  factored  into  early  planning.  Many  times  a  support  capability  will 
evolve  during  the  software  development.  Planning  should  allow  transfer 
of  such  a  capability  to  the  supporting  command  prior  to  program  manage¬ 
ment  responsibility  transfer.  The  CRISP  is  an  appropriate  document  for 
addressing  this  or  similar  items. 


CONCEPT  OF  TESTING 
TEST  ORGANIZATIONAL 
RELATIONSHIP 


TEST  EFFORT ^/SCHEDULES 
SCOPE  OF  TESTING 
INTEGRATION  TESTING 


3.2  SOFTWARE  DEVELOPMENT  PLANS 


Section  3.1  addressed  the  plans  which  outline  the  management  con¬ 
trol  of  an  acquisition  involving  software  development.  This  section  covers 
the  controls,  disciplines,  and  tests  that  pertain  directly  to  development 
of  the  software.  The  development  plans  must  interface  and  complement 
the  management  control  plans,  but  are  normally  contractor  generated  in 
coordination  with  the  Air  Force  implementing  agency. 

3.2.1  Computer  Program  Development  Plan 

Attention  must  be  given  to  the  actual  controls  and  mechanisms 
necessary  to  generate  software  programs.  The  Computer  Program  Devel¬ 
opment  Plan  (CPDP)  is  commonly  used  to  provide  this  attention.  The 
CPDP  should  be  submitted  as  a  part  of  the  precontractual  response  to 
the  Request  for  Proposal.  Submission  of  this  information  should  enable 
the  government  agency  to  gain  a  broad  insight  of  the  bidder's  capability 
to  produce  the  software.  Even  though  a  CPDP  cannot  be  prepared  com¬ 
pletely  as  part  of  a  contractor's  proposal  because  the  system  definition 
has  not  progressed  sufficiently,  the  fundamental  approach  to  computer 
program  development  should  be  documented  and  the  CPDP  updated  at 
some  later  time.  Specific  topics  to  be  contained  within  the  CPDP  are 
listed  in  AFR  800-14  and  Appendix  B  of  this  document.  Data  Item 
Description  (DID)  DI-S-30567A  contains  a  further  expanded  list  of  topics 
to  be  addressed  by  the  CPDP  and  the  DID  is  included  in  its  entirety  in 
Appendix  D. 

Some  additional  topics  which  should  be  considered  are: 

1.  What  is  the  order  of  module  development? 

2.  What  are  the  interdependencies  of  the  modules? 

3.  Are  simulations  or  emulations  necessary? 

4.  How  are  user  manuals  and  positional  handbooks  to  be 
evaluated? 

5.  Are  the  detailed  integration  and  tests  of  related  hard¬ 
ware  elements  and  modules  adequately  planned? 

Certain  software  developments  require  establishment  of  development 
facilities  to  house  required  equipment  and  personnel.  Specific  planning 
attention  must  be  devoted  to  acquiring  any  associated  long-lead  items. 
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site  construction,  and/or  concept  of  operation.  In  particular,  developments 
which  use  Government  Furnished  Equipment  (GFE)  or  government  controlled 
buildings  require  advanced  coordination  and  planning. 

Another  important  planning/control  consideration  is  the  amount  and 
types  of  software  documentation  to  be  produced.  Documentation  is  needed 
during  development  to  track  progress  and  provide  information  for  manage¬ 
ment  visibility  and  decision  making.  Subsequently,  it  is  needed  to  enable 
proper  life  cycle  support  and  maintenance  of  computer  resources.  The 
CPDP  should  describe  how  the  developing  documentation  will  be  generated. 

During  and  subsequent  to  software  development,  the  use  of  software 
will  uncover  discrepancies  and  desired  improvements.  A  mechanism 
must  be  instituted  to  allow  orderly  update.  The  Configuration  Manage¬ 
ment  Plan  (CMP)  addresses  the  management  and  administrative  control 
of  such  updates;  however,  the  CPDP  should  address  the  engineering/ 
technical  mechanisms.  In  particular,  it  should  insure  that  a  change 
mechanism  exists,  feedback  and  interaction  will  occur  among  various 
system  and  component  design  and  development  activities,  and  an  efficient 
approach  to  additional  testing  is  defined. 

3.2. 1.1  Development  Visibility/Control  Through  Use  of  the  Computer 
Program  Development  Plan 

The  CPDP  describes  the  development  methods  and  procedures  which 
the  contractor  will  use  in  developing  the  software.  Monitoring  the  con¬ 
tractor  development  progress  against  the  published  CPDP  will  allow 
appropriate  Air  Force  control  and  visibility.  Air  Force  technical  and 
management  control  procedures  should  be  aligned  to  interact  with  con¬ 
tractor  procedures  specified  in  the  CPDP. 

3.2.2  Configuration  Management  Plan 

Configuration  Management  (CM)  is  a  discipline  applying  technical 
and  administrative  direction  and  surveillance  to  identify,  document,  con¬ 
trol,  and  change  a  system.  A  Configuration  Management  Plan  is  the 
scheme  by  which  the  CM  will  be  implemented.  In  the  plan,  the  contractor 
will  describe  how  a  system  will  be  initially  baselined,  how  the  system  will 
be  documented  and  controlled  from  that  baseline,  what  the  approval  chan¬ 
nels  (internal  and  the  Air  Force)  are,  and  the  concept  for  making  changes 
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to  the  baseline.  Completion  of  the  CMP  is  important  because  it  forces  the 
contractor  to  plan  each  of  these  functions.  Implementing  the  plan  enhances 
the  probability  of  adequate  software  developed  in  a  more  efficient  manner. 
The  CMP  should  contain  the  specifics  of  configuration  identification,  con¬ 
figuration  control,  and  configuration  status  accounting.  Internal  contrac¬ 
tor  procedures,  prior  to  formal  Air  Force  baselining  are  of  particular 
importance,  since  this  may  be  the  only  document  covering  this  aspect. 

More  coverage  of  CM  is  contained  within  the  Configuration  Management 
Guidebook  in  this  series. 

3.  2.  3  Quality  Assurance  Plan 

AFR  74-1  defines  Quality  Assurance  (QA)  as  a  planned  and  systematic 
pattern  of  all  actions  necessary  to  provide  adequate  confidence  that  mate¬ 
rial,  data,  supplies,  and  services  conform  to  established  technical  require¬ 
ments  and  achieve  satisfactory  performance.  In  a  large  or  complex  soft¬ 
ware  development  within  a  weapon  system  acquisition,  it  may  be  advan¬ 
tageous  to  formally  apply  the  provisions  of  MIL-S  5277 9A,  " Software 
QA  Program  Requirements".  Where  this  is  done,  a  Software  Quality 
Assurance  Plan  (QAP),  separate  from  the  System  QAP  is  highly  desirable 
and  should  be  required  by  the  CDRL.  For  acquisitions  which  cannot  use 
MIL-S  52779A,  the  standard  still  serves  as  a  valuable  reference  source 
for  aligning  the  overall  QA  program  and  for  tailoring  of  the  QAP  Data 
Item  Descriptions  to  be  more  responsive  to  software  QA  requirements. 

The  SAE  Quality  Assurance  Guidebook  provides  explicit  coverage  on  the 
content  and  intent  of  a  QA  plan.  Quality  Assurance  is  stressed  here 
because  of  its  positive  affect  on  status  monitoring /control.  A  comprehen¬ 
sive  Software  QAP  increases  Program  Office  visibility  through  periodic 
QA  reports  and  more  effective  interface  with  AFPRO/DCAS  and  contrac¬ 
tor  quality  personnel.  Enhanced  visibility  coupled  with  a  reduced  need  to 
monitor  details,  such  as  coding  practices  and  standards,  make  a  Software 
QAP  a  significant  leverage  item  in  achieving  program  office  control  effec¬ 
tiveness.  The  QA  plan  should  be  commenced  prior  to  contract  award  so 
that  a  preliminary  version  can  be  used  as  an  input  for  the  source  selection. 

3.  3  SOFTWARE  SUPPORT  PLANS 

Computer  programs  delivered  at  the  conclusion  of  the  development 


phase  of  the  system  life  cycle  remain  evolutionary  throughout  the  useful 
life  of  the  overall  system.  Consequently,  some  method  of  supporting  the 
software  must  be  created.  The  more  complex  the  software,  the  more 
sophisticated  the  support  must  be,  both  in  terms  of  equipment  and  per¬ 
sonnel  expertise.  It  is  the  intent  of  this  section  to  stimulate  thought  and 
planning  to  enable  the  software  support  capability  to  be  acquired  and 
implemented. 

3.  3.  1  Computer  Resources  Integrated  Support  Plan 

Earlier  coverage  of  the  Program  Management  Plan  in  this  guidebook 
highlighted  the  responsibility  of  the  Program  Manager  to  see  that  a  CRISP 
is  written.  The  PM  accomplishes  this  by  forming  a  Computer  Resources 
Working  Group  (CRWG)  under  an  approved  charter.  Like  any  other  plan, 
the  CRISP  is  subject  to  change  as  its  input  data  changes.  The  CRISP  is  a 
"living  document"  because  of  the  extensive  and  constant  change  it  undergoes. 
Planning  for  support  of  computer  resources  is  commenced  at  approximately 
the  same  time  as  planning  for  the  development  of  the  computer  resources. 
Thus  the  CRWG  is  a  forum  for  assuring  that  issues  such  as  conflicts  bet¬ 
ween  the  operational  and  support  concepts,  support  responsibilities,  and 
long  lead  procurements  are  addressed  and  resolved.  This  guidebook  will 
not  attempt  to  resolve  these  issues,  but,  rather  to  lead  through  a  sequence 
of  steps  to  facilitate  adequate  planning  and  recognition  of  the  problems  such 
that  timely  decisions  can  be  made.  Appendix  C  of  this  document  contains 
a  list  of  required  CRISP  items. 

3.  3.  1.  1  Development  Visibility/Control  Through  Use  of  a  Computer 
Resources  Integrated  Support  fclan 

The  CRISP,  when  formally  approved  and  implemented  by  the  using, 
implementing,  and  supporting  commands,  should  insure  that  sufficient 
capability  is  available  at  Program  Management  Responsibility  Transfer 
(PMRT)  to  allow  operational  and  maintenance  support  of  computer  re¬ 
sources.  All  maintenance /diagnostic  software  should  have  been  developed 
and  tested  for  adequacy.  If  facilities  are  necessary  to  provide  hardware 
or  software  simulations,  these  should  be  complete  and  operational.  The 
control  gained  through  use  of  a  CRISP  is  that  no  support  equipment,  docu* 
mentation,  or  programs  are  omitted  which  would  jeopardize  software 
support. 


An  additional  concern  related  to  the  CRISP  is  the  testing  of  the  sup¬ 
porting  software  for  the  "basic  system"  software.  In  particular,  complex 
systems  which  require  an  Avionics  Integration  Support  Facility  (AISF)  can 
contain  sophisticated  software  within  the  support  capability  itself.  Typically, 
the  support  system  must:  enable  the  operational  system  to  think  it  is 
operating  in  its  normal  environment,  enable  the  operational  system  to  be 
corrected  or  changed,  provide  a  scenario  to  exercise  the  operational 
system,  provide  diagnostic  capability,  generate  any  necessary  simulations, 
exercise  models,  or  emulations,  and  provide  analytic  and  data  recording 
capabilities.  A  test  for  the  support  system  must  be  written  and  the  CRISP 
is  the  document  which  flags  the  requirement. 

3.  3.  2  Operational/Support  Configuration  Management  Procedures 

Preparation  of  an  Operational /Support  Configuration  Management 
Procedures  (O/SCMP)  is  the  responsibility  of  the  using  and  supporting 
commands.  This  document,  which  is  heavily  dependent  upon  the  CRISP, 
should  be  generated  near  the  end  of  the  development  phase  prior  to  PMRT. 
Coverage  of  the  O/SCMP  is  given  in  this  guidebook  to  assist  in  its  genera¬ 
tion  and  because  it  is  not  explicitly  addressed  in  other  guidebooks  of 
this  series. 

The  O/SCMP  contains  Configuration  Management  procedures  to  be 
used  during  the  operational  life  of  a  system.  Any  complex  system  using 
software  will  undergo  software  changes  during  its  lifetime,  and  in  par¬ 
ticular,  avionics  systems:  thus  the  CM  discipline  is  important  even  after 
development  completion.  As  a  minimum,  the  O/SCMP  should  incorpo¬ 
rate  the  provisions  of  change  control  and  the  following 

a.  The  relationship  of  all  commands  involved. 

b.  The  reporting  of  problems. 

c.  The  method  of  processing  deficiency  reports  or  modifications. 

d.  Approval  authority  for  changes. 

e.  Status  accounting  procedures  and  responsibilities. 

f.  The  handling  of  emergency  changes. 

g.  Methods  for  distribution  of  CPCI's  and  changes  thereto. 

h.  Situations  where  system  turnover  preceeds  PMRT. 
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Generally,  the  delaying  factor  in  generation  of  the  document  is  a 
lack  of  agreement  between  the  using  and  support  commands  over  the  pro¬ 
cedures  to  use  in  reporting,  processing,  and  correcting  problems.  The 
O/SCMP  simply  records,  in  procedural  format,  the  concepts  by  which 
system  deficiencies  are  identified,  reported,  and  corrected. 

3.4  FACTORS  AFFECTING  SOFTWARE  DEVELOPMENT  STATUS 

This  section  emphasizes  those  factors  which  will  impact,  either 
positively  or  negatively,  the  actual  development  progress.  It  is  under¬ 
stood  that  certain  of  these  factors  themselves  may  be  beyond  the  realm 
of  management  influence,  but  their  impact  to  the  software  development 
must  still  be  understood  and  considered. 

3.4.1  Requirements  Stability /Traceability 

More  discipline  and  structure  are  being  adopted  and  applied  to 
embedded  computer  systems  each  year.  The  government  approach  to  this 
uses  MIL -STD 483/490  to  specify  the  documentation  of  software  require¬ 
ments.  Type  A  specifications  containing  system  requirements  cause  the 
generation  of  Type  B- 5  specifications  (design-to  requirements)  followed 
by  C- 5  specifications  (code-to  requirements).  Each  is  at  a  lower  level 
of  detail  than  the  former  and  is  dependent  upon  the  quality  of  the  fore¬ 
runner  documents.  At  each  level,  a  set  of  requirements  serves  as  the 
input  and  a  design  is  produced  as  the  output.  This  design  becomes  the 
requirements  set  for  the  designer  at  the  next  level  down.  Using  this 
"top  down"  approach,  the  cascading  effect  of  a  system  requirement  change 
is  apparent.  The  degree  to  which  the  system  requirements  change  (sta¬ 
bility)  directly  affects  the  generation  of  design  and  code.  (The  underlying 
fact  is  that  Type  A,  B-5,  and  C-5  specifications  are  all  technical  require¬ 
ments.  There  should  have  been  an  earlier  interpretation  and  translation 
of  "operational"  requirements  into  technical  requirements).  Consider 
the  case  where  a  user  is  uncertain  of  his  detailed  operational  requirements. 
Uncertainty  at  the  operational  level  can  only  be  amplified  when  translated 
into  technical  requirements.  The  resulting  requirements  specification 
will  be  either  excessively  vague  or  will  not  resemble  a  solution  to  the  real 
user  need.  Also  consider  the  case  where  a  user  changes  his  operational 
requirement,  then  look  at  the  "ripple"  effect  the  change  has  depending 
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upon  whether  the  Type  A,  B-5,  and/or  C - 5  specifications  have  been  gener¬ 
ated.  Software  development  progress  is  that  rate  at  which  validated  soft¬ 
ware  programs  are  produced;  if  the  requirements  (at  any  level)  change, 
development  progress  is  impeded.  If  requirements  continually  change, 
development  progress  becomes  negative. 

In  addition  to  the  changing  requirements,  there  is  the  problem  of 
referencing  an  operational  requirement  all  the  way  into  a  set  of  computer 
programs  (traceability).  This  reference  is  essential  to  ensure  that  the 
user  is  getting  a  product  to  meet  his  needs.  The  interpretation/transla¬ 
tion  referred  to  earlier  also  affects  the  traceability  of  requirements. 
Assuming  the  user  is  not  convinced  his  need  is  being  met,  there  will  be 
a  reinterpretation,  retranslation,  redesign,  restatement  of  requirement, 
or  retest  of  the  software.  Any  of  these  will  impede  software  develop¬ 
ment  progress. 

In  summary,  the  more  stable  and  explicit  the  requirements,  the 
more  traceable  and  producible  the  software.  On  the  other  hand,  any 
complex  software  requirement  cannot  be  completely  stabilized  prior  to 
development.  Program  Managers  must  make  tradeoff  decisions  in  this 
area  to  enhance  the  probability  of  program  success.  For  example,  if  re¬ 
quirements  arp  constantly  undergoing  change,  one  possible  approach  would 
be  to  "freeze"  the  requirements  on  an  interim  basis  to  allow  design  pro¬ 
gress  against  a  specific  baseline,  then  incorporate  a  "block"  change. 

3.  4.  2  Project  Management 

"There  are  more  opportunities  for  improving  software  productivity 
and  quality  in  the  area  of  management  than  anywhere  else.  The  difference 
between  software  project  successes  and  failures  has  most  often  been  traced 
to  good  or  poor  practices  in  software  management".  * 

In  many  cases,  the  most  serious  software  development  pitfall  is  the 
inability  of  management  to  recognize  the  technical  significance  of  certain 
management  decisions.  As  an  example,  the  relaxation  of  configuration 
control  enforcement  to  allow  quicker  documentation  completion  can  assist 
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in  meeting  the  documentation  milestone,  but  can  produce  an  unuseable 
document.  This  appears  to  be  a  common  occurrence.  Testing  areas 
also  are  quite  commonly  compromised  without  full  realization  of  the  con¬ 
sequences. 

Even  the  management  structure  can  influence  development  progress. 
In  general,  the  simpler,  more  direct  control  organizations  function  more 
efficiently  in  software  development;  however,  the  degree  of  risk  of  this 
type  organization  correlates  to  the  experience  level  of  those  personnel  in 
key  positions. 

Certain  projects  are  attempted  with  multi-agency  (multi- command 
or  multi-contractor)  interests  and  personnel  directly  involved  in  the  soft¬ 
ware  development.  These  arrangements  have  a  tendency  toward  committee 
management.  Ideally,  the  committee  approach  should  more  fairly  repre¬ 
sent  parochial  agency  interests.  Realistically,  decisions  are  made  slowly 
and  resulting  delays  and  compromises  can  seriously  affect  the  usefulness 
of  the  entire  system.  In  projects  of  this  sort,  it  is  suggested  that  a  very 
narrow  focal  point  of  management  control  be  established  with  "near  dic¬ 
tatorial"  authority  for  final  decisions. 

3.  4.  3  Project  Complexity 

At  the  most  fundamental  level,  software  complexity  is  a  composite 
of  only  two  basic  considerations  -  control  complexity  requirements  and 
data  manipulation  requirements.  Control  complexity  might  be  further 
factored  into  timing,  accuracy,  and  interface  requirements.  How  and  to 
what  extent  these  contribute  and  interact  to  yield  some  overall  complexity 
is  currently  one  of  the  most  active  areas  in  software  research.  Such 
efforts  as  cost  and  error  prediction  models  are  examples.  For  the  pres¬ 
ent  however,  the  normal  method  of  estimating  software  complexity  must 
rely  on  comparison  to  previously  developed  similar  systems,  plus  a  cer¬ 
tain  amount  of  intuition.  Section  3.4  is  intended  to  help  form  at  least  a 
part  of  this  intuitive  estimate# 
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3.4.4  System  Interfaces 

In  some  respects,  the  complexity  of  a  system  is  related  to  the  num¬ 
ber  of  interfaces  the  system  has  with  other  systems.  Avionics  programs, 
such  as  that  of  a  central  computer  aboard  an  airplane,  interface  with  many 
other  systems.  Information  flow  through  some  of  the  interfaces  is  two-way, 
while  others  are  only  one-way.  In  any  case,  the  processing  sequence  and 
capacity  are  more  severely  taxed  by  the  complex  systems  (many  interfaces) 
than  the  simpler  systems.  It  naturally  follows  that  development  of  the  more 
complex  programs  proceeds  slower  than  the  simple  ones. 

3.  4.  5  Data  Base  Management 

Any  large  or  complex  software  system  is  likely  to  include  extensive 
amounts  of  data.  Organization  of  the  data  into  various  storage  areas,  in 
particular  as  unreferenced  data  (files)  or  cross  referenced  data  (data 
base),  requires  vastly  different  considerations  in  designing  the  scheme  to 
manipulate  and  use  the  data.  If  the  computer  program  to  be  developed  re¬ 
quires  a  sophisticated  data  management  scheme,  development  will  require 
more  time  than  a  simpler  system. 

3.4.6  Constraints 

Several  common  system  constraints  are  discussed  in  this  section  to 
indicate  the  kinds  of  impact  to  software  development  that  they  may  cause. 
Quite  often,  projects  are  attempted  with  compressed  development  schedules, 
processor  limitations,  memory  limitations,  inadequate  peripherals  and 
tools,  inadequate  facility  support,  inadequate  manning,  or  inadequate 
expertise.  Any  of  these  can  be  detrimental  to  development  progress  even 
if  management  is  aware  of  them  and  otherwise  adequate  methods  and 
procedures  are  used. 

Compressed  schedules  have  a  tendency  to  encourage  planning  cur¬ 
tailment  and  a  relaxation  of  engineering  controls.  In  other  words,  the  pro¬ 
ject  becomes  milestone  driven  instead  of  quality,  end-product  driven.  As 
the  pressure  of  meeting  the  compressed  schedule  increases,  more  and 
more  compromise  of  planning  and  engineering  controls  become  evident. 

The  danger  of  not  adequately  implementing  these  management  functions  is 
evident  and  the  results  are  negative.  A  school  of  thought  exists  which  says, 
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"If  you  are  behind  schedule,  add  more  manpower.  "  This  is  analogous  to 
attempting  to  put  out  an  oil  fire  with  water.  Each  person  added  must 
become  acquainted  with  the  system  as  well  as  his  assigned  task.  This, 
when  coupled  with  the  hurry  up  mode,  only  compounds  the  management 
control  problems. 

Processor  and  memory  limitations  are  somewhat  related  in  their 
effects  on  development  progress.  Violation  of  either  limit  requires  some 
workaround  to  allow  an  apparently  improved  processor  or  larger  memory. 
Workarounds,  by  their  nature,  are  difficult  to  document  and  are  usually 
done  so  inadequately.  More  programming  (design  and  code)  is  required  to 
use  roll-in,  roll-out  schemes  to  allow  more  room  in  memory.  These 
added  programs  especially  aggravate  system  timing  (sort  of  a  Program 
Evaluation  Review  Technique  (PERT)  consideration  for  the  system  elements) 
and  increase  design  complexity.  Processor  limitations  are  sometimes 
helped  by  rewriting  programs  to  run  faster.  This  usually  means  the  code 
is  more  interdependent,  complex,  and  susceptible  to  error.  Trying  to 
"shoehorn"  more  code  into  a  full  memory  or  make  a  processor  "work 
faster"  will  be  detrimental  to  the  basic  software  development  progress. 
Project  specifications  should  require  spare  processor  capability  and 
memory  proportional  to  the  technical  risk  of  the  software.  Trade  studies 
during  validation  phase  are  the  appropriate  vehicle  to  address  the  quan¬ 
tification  of  the  spare  capability. 

Large  software  developments  usually  require  capable  processing 
centers  equipped  with  high  speed  printers,  display  terminals,  tape  units, 
discs,  etc.  Additionally,  software  tools  such  as  trace,  dump,  sort  rou¬ 
tines,  etc.  are  extensively  used.  Occasionally,  a  development  is  attempted 
which  has  a  scarcity  of  peripherals,  tools,  or  even  the  basic  facilities  to 
support  the  development.  This  results  in  some  negative  development 
impact  while  workarounds  are  developed  or  substitutes  are  used.  Such  con¬ 
siderations  as  remote  locations  of  coders  from  the  processing  center, 
restricted  hours  of  operation,  classified  processing  requirements,  excessive 
downtime,  etc. ,  can  delay  the  completion  of  programs  or  modules  on 
which  others  are  dependent.  All  of  these  scarcities  can  cause  develop¬ 
ment  impacts  of  various  magnitudes. 
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The  range  of  operating  systems  available  today  is  enormous,  but  all 
fit  into  the  category  of  batch-processing  and/or  interactive.  The  inter¬ 
active  system  allows  the  programmer,  through  use  of  a  display  or  terminal, 
to  communicate  directly  with  the  computer  while  the  batch -processing 
system  isolates  the  programmer  from  the  computer.  Batch-processing 
systems  are  most  often  found  in  computing  centers  where  program  develop¬ 
ment  is  considered  a  minor  part  of  the  overall  activity,  and  efficient  pro¬ 
duction  runs  are  considered  a  major  part.  Interactive  systems  are  often 
found  in  centers  where  program  development  is  the  primary  activity  and 
where  the  nature  of  the  problems  being  solved  dictate  programmer /com¬ 
puter  interaction. 

One  of  the  most  crucial  constraints  to  developing  software  relates 
to  the  adequacy  of  manning  and  the  expertise  of  personnel.  It  is  very 
risky  to  rely  upon  too  few  people,  but  there  is  also  a  significant  manage¬ 
ment  problem  in  using  too  many  people.  Attempting  to  use  new  or  unin¬ 
formed  personnel  means  that  a  learning  process  must  occur  prior  to  their 
being  productive.  For  complex  systems,  the  learning  curve  can  be  up  to 
one  year.  Schedules  should  take  into  consideration  the  estimated  produc¬ 
tivity  rate  of  the  various  developers.  Unfortunately,  no  magic  key  exists 
which  enables  the  manning  of  particular  tasks  at  particular  levels  with 
particular  skills  to  provide  efficient  software  development.  The  general 
rule  is  to  assign  more  experienced  and  capable  personnel  to  design 
related  tasks  and  integration  and  lessknowledgable  people  to  lesser  tasks. 
Development  progress  relates  in  an  intangible  way  to  personnel  compe¬ 
tency,  experience,  and  manning  levels.  Development  progress  is  maxi¬ 
mized  when  these  areas  are  maximized. 

3.4.7  Degree  of  Modularity 

A  program  module  can  be  defined  as  a  logically  self-contained  and 
discrete  part  of  a  larger  program.  It  accepts  well-defined  inputs,  uses 
well-defined  processing,  and  produces  outputs  of  well-defined  content. 
Modular  design  should  break  complex  tasks  into  smaller  and  simpler 
subtasks  which  enhance  the  probability  of  writing  more  correct  programs. 
This  implies  a  significant  increase  of  effort  during  the  preliminary 
design  phase  of  software  production  to  achieve  payoff  during  code  and  test 


phases.  The  degree  of  modularity  is  the  extent  to  which  a  large  set  of 
programs  can  be  conceived  as  made  up  of  a  composition  of  smaller  pro¬ 
gram  elements.  Largely  modular  programs  lend  themselves  to  simpler 
design  and  less  interactions  between  program  parts,  thus  simplifying  the 
overall  program. 

3.4.8  Programming  Languages,  Standards  and  Practices 

The  applicability  of  the  language(s)  to  the  desired  software  develop¬ 
ment  and  the  discipline  of  using  approved  programming  practices  both  have 
positive  effects  upon  development  progress.  The  incorporation  of  stan¬ 
dards  for  programming  will  also  save  time  because  it  allows  a  more 
orderly,  understandable  progression  through  these  design,  code,  and  test 
activities.  Poor  practices  or  standards  omissions  often  result  in 
reaccomplishment  to  provide  a  basis  of  understanding  for  computer 
program  maintenance  and  further  enhancement. 

3.4.9  Uniqueness 

Development  progress  of  software,  which  is  totally  unlike  any  pre¬ 
viously  generated  program,  is  slower  than  "variations  on  a  familiar  theme". 
Care  should  be  exercised  to  avoid  pushing  beyond  the  state-of-the-art  and 
in  areas  where  few  have  previously  been.  Technical  risk  goes  up  propor¬ 
tionate  to  the  deviation  from  previously  developed  program  designs.  New 
or  one  of  a  kind  hardware  systems  which  require  software  development  are 
also  prone  to  be  associated  with  lengthy  development  schedules. 

3.4.10  Security 

The  coverage  of  security  in  this  section  is  not  intended  to  reflect 
guidance  additional  to  DOD  and  Air  Force  regulations  but  to  emphasize 
the  additional  impacts  of  developing  software  which  contains  classified 
programs  or  data.  There  are  three  types  of  security  considerations  which 
will  affect  development  progress. 

Production  of  software  containing  classified  data  means  that  physical 
security  procedures  must  be  implemented,  to  limit  access  to  the  facilities 
to  only  those  appropriately  cleared  people  who  have  a  need -to- know.  Com¬ 
puter  facilities  and  procedures  must  be  established  which  restrict  unautho¬ 
rized  compromise  or  exposure  of  classified  data. 
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A  second  consideration  is  associated  with  the  actual  generation  of 
the  data.  Some  method  of  controlled  data  acquisition  (even  if  it's  only- 
classified  mail)  must  be  defined  and  implemented.  Appropriate  classifica¬ 
tion  of  actual  "generated"  data  must  be  established  in  accordance  with 
DOD  and  Air  Force  regulations. 

The  third  consideration  is  control  of  access  to  the  classified  data 
base.  Programmers  or  users  of  sensitive  data  must  be  assured  that 
unauthorized  personnel  cannot  intentionally  nor  inadvertently  access  the 
classified  data.  Protective  software  and/or  procedure  must  be  imple- 
mented  to  preclude  this  activity# 

3.5  STATUS  QUANTIFICATION 

Several  attempts  at  quantifying  software  development  progress  have 
been  made,  some  of  which  have  resulted  in  models,  formulas,  procedures, 
etc.  The  problem  of  exact  status  is  a  complicated  one  in  which  a  truly 
generalized  coverage  is,  at  this  point,  unknown.  Assuming  that  it  is 
possible  to  forecast  the  approximate  number  of  instructions  to  make  up  a 
software  module,  when  a  certain  percentage  of  the  code  is  done,  a  certain 
quantified  portion  of  the  module  is  done.  However,  the  module  isn't  yet 
tested,  nor  is  it  integrated  with  other  modules  and  tested.  If  one  could 
go  through  a  module  code  generation  only  once,  test  the  results  with  no 
rewrite,  and  be  assured  the  module  would  properly  interface  with  others, 
then  completion  of  a  percentage  of  the  code  would  equate  to  a  specific 
percent  of  completion.  Experience  indicates  that  some  degree  of  rewrite 
is  highly  probable.  There  will  be  module  interface  problems  which  could 
perturbate  basic  design,  and  there  is  likely  to  be  some  testing  which  requires 
retest.  This  is  not  to  imply  that  counting  lines  of  code  is  a  useless 
exercise,  but  simply  that  an  exact  status  will  not  be  indicated.  If  the 
module  in  question  represents  30  percent  of  the  total  task  and  50  percent 
has  been  coded  and  tested,  then  you  are  somewhere  below  15  percent 
project  complete.  At  least  there  is  some  indication  of  upper  bound. 

Another  common  approach  to  quantification  of  status  is  by  "functional 
threads.  "  By  this  theory,  a  function  starts  in  some  module  and  threads 
its  way  through  one  or  more  additional  modules.  The  inputs  to  the  func- 
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tional  thread  are  supplied  and  the  reaction  of  the  functional  thread  is 
checked.  When  the  results  test  o.k.  ,  this  functional  thread  is  completed. 
If  you  assume  this  thread  makes  up  some  percentage  of  the  total  effort, 
then  upon  its  completion,  you  are  that  percent  complete  of  the  total  effort. 
Unfortunately,  there  are  few  (if  any)  functional  threads  which  are  com¬ 
pletely  independent  of  all  other  threads.  Therefore,  any  influence  or 
feedback  of  the  other  threads  on  the  one  just  completed  is  discounted  in 
indicating  status.  Like  the  module  sizing  theory,  the  thread  theory  will 
give  some  indication  of  development  status  but  not  an  exact  indication. 

Note  that  both  of  the  theories  discussed  in  this  section  indicate  a 
reliance  upon  modularly  designed  software.  An  unmodularized  design 
would  not  lend  itself  to  realistic  indications  using  either  theory. 


-  29  - 


4.  MONITORING  AND  CONTROLLING  DEVELOPMENT  STATUS 


The  discussion  in  Section  3  emphasized  the  necessity  for  proper 
planning  of  a  software  development.  This  chapter  emphasizes  the  need 
for  controlling  against  the  plan  and  the  mechanisms  considered  are 

e  Milestones  and  Schedules 

s  Reviews  and  Meetings 

s  Monitoring  Development  Testing 

#  Contractor  Progress  Reports 

s  Financial  Performance  Reports 

e  Use  of  IV&V  Contractor 

4.1  MILESTONES  AND  SCHEDULES 

DODD  5000. 1  identifies  four  key  milestones  for  major  System 
Acquisitions  as  being  of  interest  to  the  Secretary  of  Defense  and  the 
Defense  Systems  Acquisition  Review  Council  (DSARC).  These  are: 

Program  Initiation,  Milestone  0;  Demonstration  and  Validation,  Milestone  I; 
Full  Scale  Engineering  and  Development,  Milestone  II;  and  Production 
and  Depldyment,  Milestone  III.  The  DOD  is  interested  in  monitoring  any 
project  at  this  level.  From  that  level  downward,  there  is  a  cascading 
effect  of  more  milestones  at  increasing  frequency  depending  upon  the 
level  of  visibility  the  Air  Force,  the  Command,  the  Program  Manager, 
the  system  engineer,  the  module  designer,  etc.,  needs.  At  the  lower 
levels,  the  relevant  milestones  may  approach  almost  a  daily  basis,  but, 
in  any  case,  they  are  designed  to  provide  assurance  that  each  level  of 
tasks  can  be  made  to  fit  the  critical  milestones  of  higher  level  schedules. 

As  discussed  in  Section  3,  exact  status  of  software  development  is  seldom 
known.  This  results  in  detailed  software  milestones  being  projected 
with  less  precision  than  might  be  desirable.  However,  these  milestones 
are  not  unuseable  if  viewed  from  the  proper  perspective  (If  an  auto  driver 
misjudges  the  distance  remaining  to  a  faraway  location  by  a  hundred 
feet,  it  is  virtually  meaningless,  but  if  he  misjudges  the  car  immediately 
in  front  of  him  by  that  amount,  it  could  be  disastrous).  The  point  is  that 
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the  milestone  increments  one  needs  are  a  function  of  not  only  the  next 
critical  decision  point,  but  also  the  level  of  confidence  with  which  the 
status  is  known.  Lower  confidence  should  result  in  more  milestones , 
even  if  they  tend  to  be  artificial. 

4.2  REVIEWS  AND  MEETINGS 

This  section  addresses  milestones  in  the  context  of  formal  reviews 
required  by  the  contract  and  informal  meetings  which  normally  occur  during 
the  software  development  process.  The  formal  milestones  to  be  addressed 
(System  Requirements  Review,  System  Design  Review,  Preliminary  Design 
Review,  Critical  Design  Review,  and  sometimes.  Test  Readiness  Review), 
are  well  defined,  highly  visible  events  upon  which  the  entire  system  develop¬ 
ment  has  been  based.  They  are  therefore  critical  points  and  must  be 
approached  with  a  high  confidence  level.  The  informal  meetings  can  be 
viewed  as  the  preliminary  milestones  which  occur  as  often  as  necessary 
to  insure  the  integrity  of  the  major  milestones.  These  may  be  called 
Technical  Interchange  Meetings  (TIM),  Management  Strategy  Meetings,  etc. 
They  may  be  periodically  scheduled  or  de  facto  meetings  called  by  the 
Program  Office  or  contractor,  but  their  importance  in  maintaining  program 
office  visibility  and  control  cannot  be  overemphasised. 

Events  leading  up  to  major  reviews  and  criteria  for  passage  are  dis¬ 
cussed  in  detail  in  the  SAE  Reviews  and  Audits  Guidebook  and  are  briefly 
outlined  below. 

4.2.1  System  Requirements  Review 

MIL-STD  1521A  indicates  the  System  Requirements  Review  (SRR) 
may  be  held  at  any  time  during  system  conceptual  and/or  validation  phases 
but  normally  after  the  accomplishment  of  functional  analysis  and  preliminary 
requirements  allocation.  Its  purpose  is  to  determine  initial  direction  and 
progress  of  the  contractor's  System  Engineering  and  Management  effort 
and  his  convergence  upon  an  optimum  and  complete  configuration.  The 
total  System  Engineering  and  Management  activity  and  its  output  are 
reviewed  for  responsiveness  to  the  Statement  of  Work  and  system  require¬ 
ments.  The  SRR  emphasizes  weapon  system  level  requirements;  however, 
where  significant  software  development  is  involved  it  is  recommended  that 
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a  separate  system  level  software  review  be  conducted,  either  independently 
or  as  a  formal  portion  of  an  SRR .  The  purpose  of  the  additional  software 
review  is  to  arrive  at  a  joint  Contractor  and  Air  Force  understanding  of 
the  provisions  stated  as  software  requirements.  Analysis  of  all  software 
requirements  on  an  individual  and  composite  basis  is  necessary  prior  to 
the  software  review.  Techniques  and  tools  used  to  perform  the  analysis 
should  be  explained,  all  identified  problem  areas  discussed,  and  remedial 
actions  outlined.  The  end  result  of  the  software  review  is  a  conceptual 
agreement  on  the  software  requirements  which  provides  the  basis  for 
development  of  the  software  requirements  specification(s). 

4.2.2  System  Design  Review 

MIL-STD  1521A  indicates  the  System  Design  Review  (SDR)  shall  be 
conducted  to  evaluate  the  optimization,  traceability,  correlation,  complete¬ 
ness,  and  the  risk  of  the  allocated  requirements  in  fulfilling  the  functional 
configuration  baseline.  The  SDR  represents  the  first  technical  review  of 
a  contractor's  progress.  Principally,  this  review  is  to  prove  the  contractor's 
understanding  of  the  requirements  and  for  the  Air  Force  to  evaluate  the 
contractor's  system  design  approach.  Specific  computer  programming 
requirements  topics  to  be  addressed  are  listed  in  Appendix  B,  Paragraphs 
20.3.21,  20.3.2m,  20.3.10k,  and  20.  3.12.1  through  20.  3. 12. 13  of  MIL-STD 
1521A. 

4.2.3  Preliminary  Design  Review 

MIL-STD  1521A  indicates  the  Preliminary  Design  Review  (PDR)  shall 
be  a  formal  technical  review  of  the  basic  design  approach  for  a  Configuration 
Item  (Cl)  or  for  a  functionally  related  group  of  Cl's.  Specifically,  avail¬ 
ability  of  the  Part  I  specification  and  accomplishment  of  the  preliminary 
design  are  necessary  prerequisites  of  the  PDR.  During  this  review,  the 
contractor  should  demonstrate  he  is  ready  to  proceed  with  the  detailed 
design.  Problem  areas  and  possible  solutions  should  be  presented  by  the 
contractor.  Specific  computer  program  configuration  items  to  be  considered 
are  addressed  in  Appendix  C,  Paragraphs  30.3.2.3,  30.3.13.4,  and  30.2.2(a) 
through  (k)  of  MIL-STD  1521A. 
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4.2.4  Critical  Design  Review 


MIL-STD  1521A  indicates  the  Critical  Design  Review  (CDR)  shall 
be  conducted  on  each  Cl  prior  to  fabrication/production  design  release  to 
insure  design  solutions  in  the  draft  Part  II  specification  satisfy  performance 
requirements  stated  in  the  Part  I  specification.  The  contractor  should 
present  his  design  approach  through  detailed  flowcharts  and  briefings  to 
the  Air  Force  at  the  proper  level  of  detail  as  implied  by  MIL-STD  1521A, 
Appendix  D.  The  CDR  is  a  review  of  the  design  prior  to  release  to  the 
coders. 

4.  2.  5  Test  Readiness  Review 

The  Test  Readiness  Review  (TRR)  is  an  informal  review  commonly 
used  by  the  development  contractor  to  review  development  test  results 
and  evaluate  preparations  for  qualification  and/or  acceptance  testing. 
Completion  of  this  review  ascertains  the  adequacy,  traceability,  and 
completion  of  accomplished  development  testing  for  planned  qualification/ 
acceptance  testing  plus  the  TRR  evaluates  the  testing  procedures,  tools/ 
facilities,  personnel,  configuration  control  procedures,  etc. ,  to  be  used 
in  the  quality  or  acceptance  tests.  Although  not  defined  in  MIL-STD  1521  A, 
this  review  has  proven  extremely,  useful  for  programs  involving  complex 
or  extended  testing  programs.  Formalizing  the  TRR  by  including  it  in 
the  Contract  Statement  of  Work  and  other  contractual  documents  should 
be  seriously  considered. 

4.  3  DEVELOPMENT  TESTING 

In  addition  to  formal  and  informal  reviews,  a  very  useful  method  for 
gauging  development  progress  is  to  monitor  software  development  testing, 
(i.e. ,  lower  levels  of  testing  than  CPCI  level).  Typically,  the  contract 
will  require  demonstrations  and  formal  acceptance  tests  at  the  CPCI  level, 
but  for  informative  purposes  tests  at  much  lower  levels  need  to  be  wit¬ 
nessed  on  a  sample  basis.  Reports  resulting  from  these  tests  should  be 
described  in  the  Computer  Program  Development  Plan  and  available  to 
the  Air  Force  if  necessary  through  the  data  accession  technique  (see  SAE 
Documentation  Requirements  Guidebook,  section  3.  3.  4.  2a.  ) 
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The  lowest  levels  of  development  testing  begin  with  individual  pro¬ 
grammers  to  insure  that  programmed  code  is  valid  and  contributes  to 
some  requirement.  Testing  at  the  CPC  and  module  level  can  also  be 
accomplished  followed  by  integration  testing  of  multiple  modules.  Finally, 
testing  of  each  CPCI  can  be  accomplished  followed  by  integration  testing 
of  multiple  CPCI' s. 

Each  level  of  testing  must  be  accompanied  by  a  comparable  level  of 
test  requirements.  In  other  words,  if  the  project  is  contracted  at  a  CPCI 
level  of  testing,  but  development  testing  is  to  be  accomplished  at  a  lower 
level,  then  test  requirements  must  be  defined  for  a  commensurate  lower 
level;  as  an  example,  testing  a  "unit"  of  software  must  be  accomplished 
via  "unit  test"  requirements. 

Subsequent  to  satisfactory  testing  of  low  level  units  of  software, 
the  tested  units  must  be  placed  under  configuration  control.  The  more 
frequent  testing  (at  lower  levels)  requires  a  greater  demand  on  configura¬ 
tion  control. 

Testing  at  any  level  is  likely  to  be  more  thorough  if  performed  by 
a  group  other  than  the  software  developing  group.  A  mechanism  for  cor¬ 
rection  of  discovered  deficiencies  must  be  implemented  to  allow  correc¬ 
tion  of  the  deficiencies. 

In  summary,  development  testing  provides  good  visibility  into  the 
actual  development  progress.  The  assistance  gained  is  directly  proportional 
to  the  quality  of  planning  preceding  the  software  unit  definitions  and  their 
interfaces. 

4.  3.  1  Test  Report 

The  test  report  details  the  results  of  the  executed  test,  ft  will 
identify  the  purpose  and  nature  of  test,  will  describe  in  detail  any  deviations 
from  the  test  specification  or  procedures,  and  will  compare  the  test 
results  with  the  expected  output. 

4,  4  CONTRACTOR  PROGRESS  REPORTS 

Another  technique  for  monitoring  development  status  is  the  formal 
report  prepared  by  the  contractor.  These  may  include: 
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•  Periodic  status  reports 

•  Analytical  reports 

•  Test  reports 

•  Meeting  reports 

There  are  multiple  formats  available  for  each  of  these  and  more  in 
the  AMSDL.  However,  in  the  case  of  formal  reports,  a  minimum  number, 
carefully  chosen  and  tailored,  if  necessary,  is  the  best  approach.  They 
tend  to  be  costly  not  only  in  dollars,  but  in  use  of  human  resources  both 
for  the  contractor  and  the  Program  Office,  proper  procedure  would  be 
to  require  those  reports  that  supply  the  data  required  for  visibility, are 
written  in  a  useful,  clear  format,  and  are  generated  at  a  time  in  the  project 
to  allow  timely  decisions  and/or  forecast  future  development  progress. 

A  periodic  report  is  submitted  at  regular  intervals  and  usually 
addresses  milestones,  accomplishments,  future  goals,  and  problem  areas. 
Analytical  reports  are  submitted  as  a  result  of  some  special  activity 
completion  and  usually  identify  study  results,  modeling,  or  simulations. 
Test  reports  follow  the  completion  of  the  appropriate  testing  activity 
and  usually  indicate  the  degree  of  requirements  verification,  data  collected, 
and  analysis  conducted.  Meeting  reports  describe  the  discussions  held 
during  a  meeting  and  any  guidance,  agreement,  or  conclusion  reached. 

4.  5  FINANCIAL  PERFORMANCE  REPORTING 

Overall,  schedule  performance  can  be  viewed  in  dollar  terms  by  com¬ 
paring  budgets  for  completed  work  to  budgets  for  scheduled  work.  The  best 
place  to  measure  accomplishment  is  at  the  level  where  work  is  performed. 
Summaries  of  Work  Breakdown  Structure  (WBS)  elements  can  provide 
meaningful  data  which  justify  management  adjustments.  If  the  software 
development  can  be  divided  into  WBS  elements,  then  actual  development 
can  be  related  to  budgeted  development.  The  management  system  which 
produces  performance  reports  must  be  reasonably  well  disciplined  to  be 
effective.  Arbitrary  transfers  of  budget  from  one  task  to  another,  for 
example,  can  destroy  the  significance  of  reported  values. 

The  Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  established 
many  of  the  characteristics  and  disciplines  required  of  an  effective 
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performance  measurement  system.  These  criteria  do  not  impose  any 
specific  management  technique  or  methodology,  but  represent  basic 
principles  applicable  to  most  management  systems.  Failure  to  meet  a 
criterion  generally  indicates  a  weakness  in  a  Cost/Schedule  Control  System. 
While  compliance  with  the  C/SCSC  is  mandatory  on  major  defense  programs, 
the  criteria  also  provide  a  useful  checklist  for  evaluating  programs  of  any 
size. 

The  Cost /Schedule  Status  Report  (C/SSR)  provides  for  reporting  the 
key  data  elements  described  by  WBS  elements  and  narrative  comments 
explaining  cost  and  schedule  variances  which  exceed  specified  thresholds. 

4.  6  INDEPENDENT  CONTRACTOR 

Independent  Verification  and  Validation  (IV&V)  is  often  used  as  a 
management  tool  for  controlling  software  development.  To  illustrate  the 
thoroughness  of  development  status  coverage,  the  following  reports  nor¬ 
mally  associated  with  IV &V  are  listed: 

•  Risk  and  criticality  study  reports 

s  Requirements  analysis  reports 

s  Design  analysis  reports 

s  Code  analysis  reports  /data  reduction 

•  V&V  test  plans  and  procedures 

a  Automated  testing  and  analysis  reports 

•  Independently  generated  scripts  and  scenarios 

•  Interface  analysis  reports 

•  Model  and  simulation  documentation  and  user  reports 

•  Error  analysis  and  trouble  reports 

•  Independent  evaluation  of  developer's  test  program 

•  Reports  of  special  studies 

s  Validated  requirements  data  base 

•  System  Performance  Analysis  Report 

The  philosophy  behind  these  V&V  products  is  to  establish  a  pipeline 
through  which  information  is  quickly  passed  to  management  with  formal 
follow-up  procedures  to  ensure  the  problem  and  analysis  reports  include 
the  disposition  and  status  of  all  issues  affecting  the  development. 
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5.  GOVERNMENT  REPORTING 

The  requirements  for  the  Program  Manager  to  report  project  status 
are  frequent  and  may  be  assessed  at  varied  levels  within  the  Air  Force 
structure.  If  a  software  problem  or  assessment  is  to  be  included  as  a 
portion  of  the  status  then  the  responsible  software  development  manager 
should  be  aware  of  the  expectations  of  these  status  requirements.  Typi¬ 
cally,  the  status  data  are  contained  in  a  few  viewgraphs  which  are  orally 
presented  by  the  Program  Manager  or  his  appointed  representative.  AFSCR 
800-1  defines  three  levels  of  status  reports:  Command  Assessment  Review 
(CAR)  to  be  given  at  Air  Force  command  level.  Program  Assessment  Review 
(PAR)  to  be  given  at  Air  Staff  levels,  and  Secretary  Program  Review 
(SPR)  to  be  given  at  Air  Force  Secretarial  levels.  Specific  guidance 
relative  to  viewgraph  format  and  topics  required  is  defined  in  "Revision 
No.  3,  Format  and  Instructions  for  SPR,  PAR,  and  CAR.  "  This  document 
published  in  April  1979,  is  supplemented  by  several  amplifying  messages 
and  explanatory  data.  Both  AFSCR  800-2  and  "Revision  No.  3"  should  be 
consulted  prior  to  completing  any  status  submittals  on  software  development. 
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APPENDIX  A 


PROCUREMENT  PLAN  TOPICS 

The  following  outline  is  offered  as  a  guide  for  topics  to  be  addressed 
by  a  Procurement  Plan  (PAR  1-2102  contains  this  guide  and  also  authorizes 
its  retailoring  to  more  specifically  address  a  particular  program  procurement). 

1.  A  description  of  the  program,  item  or  system 

2.  Program  Funding  (R&D  and  Production)  including  a  summary 
of  monies  in  the  FYDP- Budget  Submissions 

3.  Delivery  Requirements,  both  R&D  and  Production  Contracts 

4.  Applicability  of  a  Decision  Coordinating  Paper  or  Program 
Memorandum  Defense  System  Acquisition  Review  Council 
(DSARC)  or  Internal  Service  Reviews 

5.  Background  and  Procurement  History 

6.  Discussion  of  Program  Risk,  including  Technical,  Cost, 
and  Schedule  Risk 

7.  Integrated  Logistics  Support  Planning  Concept 

8.  Application  of  Design  to  Cost 

9.  Application  of  Life  Cycle  Cost 

10.  Reliability  and  Maintainability  Objective 

11.  Test  and  Evaluation  Approach 

12.  Approval  for  Operational  Use 

13.  Government  Furnished  Material/Facilities/Component 
Breakout 

14.  Application  of  Should  Cost 

15.  Milestone  Chart  Attachment  Depicting  the  Objectives  of 
the  Acquisition 

16.  Milestones  for  Updating  the  Procurement  Plan 

17.  Identification  of  Participants  in  the  PP  Preparation 

18.  Procurement  Approach  for  each  Proposed  Contract 
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COMPUTER  PROGRAM  DEVELOPMENT  PLAN  TOPICS 


The  CPDP  identifies  the  actions  needed  to  develop  and  deliver  computer 
program  configuration  items  and  necessary  support  resources.  Specifically, 
AFR  800-14  requires  the  plan  to  address  the  following 

a.  The  organization,  responsibilities  and  structure  of  the 
group(s)  that  will  be  designing,  producing,  and  testing 
all  computer  programs. 

b.  The  management  and  technical  controls  that  will  be 
used  during  the  development,  including  controls  for 
insuring  that  all  performance  and  design  requirements 
have  been  implemented. 

c.  The  methodology  for  insuring  satisfactory  design  and 
testing,  including  quality  assurance. 

d.  The  development  schedule  for  each  computer  program 
configuration  item  and  proposed  program  milestone 
review  points. 

e.  The  procedure  for  monitoring  and  reporting  the  status 
of  computer  program  development. 

f.  The  resources  required  to  support  the  development 
and  test  of  computer  programs.  Special  simulation, 
data  reduction  or  utility  tools  that  are  planned  for 

use  in  the  development  of  computer  programs  should  be 
identified. 

g.  The  general  procedures  for  reporting,  monitoring,  and 
resolving  computer  program  errors  and  deficiencies 
during  development  and  testing. 

h.  The  methods  and  procedures  for  collecting,  analyzing, 
monitoring,  and  reporting  on  the  timing  of  time  critical 
computer  programs. 

i.  The  management  of  computer  program  development  masters, 
data  bases,  and  associated  documentation  including  its 
relationship  to  the  configuration  management. 

j.  Guidelines  and  checkpoints  for  insuring  future  computer 
program  growth,  modularity,  and  ease  of  modification. 

k.  The  approach  for  developing  computer  program  documenta¬ 
tion, 

l.  Training  requirements  and  associated  equipment  for  the 
deployment  phase. 


m.  Engineering  practices  include  standards,  conventions, 
procedures,  rules  for  program  design;  program  struc¬ 
tures  and  conventions;  display  and  logic  standards; 
input /output  signal  standards;  and  other  disciplines 
affecting  development. 

n.  Security  controls  and  requirements. 

o.  Simulation  techniques  and  tasks. 
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APPENDIX  C 


COMPUTER  RESOURCES  INTEGRATED  SUPPORT  PLAN  ITEMS 


The  Computer  Resources  Integrated  Support  Plan  (CRISP)  identifies 
organizational  relationships  and  responsibilities  for  the  management  and 
technical  support  of  computer  resources*  It  functions  during  the  full-scale 
development  phase  to  identify  computer  resources  necessary  to  support 
computer  programs  after  transfer  of  Program  Management  Responsibility 
and  system  Turnover  (PMRT).  The  CRISP  continues  to  function  after  PMRT 
as  the  basic  agreement  between  the  supporting  and  using  commands  for 
management  and  support  of  computer  resources*  AFR  800-14  requires  the 
following  items  to  be  included,  as  applicable 

a*  Offices  of  primary  responsibility  and  management  focal 
points  for  support  of  computer  resources  and  the  channels 
of  communication  among  organizations. 

b.  Planning  for  configuration  management  of  computer  programs 
including  the  assignment  of  configuration  control  responsibilities 
during  the  deployment  phase.  This  planning  will  reflect  the 
operational  and  support  concepts  for  the  system. 

c.  Responsibilities  for  composite  system  integrity  include 

1.  Computer  storage  utilization 

2.  Computer  program  operating  time  and  priorities 

3.  Computer  program  interface  techniques 

4.  Computer  program  baseline  integrity 

5*  Utilization  of  computer  modules  and  peripherals 

d.  Documentation  required  to  support  each  type  of  computer 
program* 

e.  Responsibility  for  funding,  scheduling,  and  system 
integration* 

f*  Qualified  personnel  required  for  supporting  computer 

equipment  and  computer  programs  together  with  training 
requirements* 

g*  Computer  equipment  and  devices  required  to  facilitate 
computer  program  changes,  including  acquisition 
responsibilities* 
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Computer  programs  required  to  support  computer  equipment 
and  other  computer  programs  including  acquisition  responsi¬ 
bilities. 

Verification  and  Validation  (VfcV)  of  computer  programs. 

Plans  to  establish  and  operate  necessary  support  facilities. 
Common  and  existing  facilities  will  be  used  whenever  practi¬ 
cable.  The  size  and  scope  of  the  support  facility  will  be  based 
on  work  load  predictions. 

Provisions  for  the  transfer  of  program  management  responsi¬ 
bility. 

Provisions  for  system /equipment  turnover. 
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Computer  Program  Development  Plan  (CPDP) 
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The  CPDP  Is  a  document  In  which  the  contractor  describes  his 
specific  detailed  plan  for  the  management  and  development  of 
all  of  the  computer  programs  and  associated  documentation 
that  he  needs  for  the  completion  of  the  contract;  this  may 
Include  support  software  such  as  simulations  and  analysis 
tools,  as  well  as  operational  programs  and  automatic  test 
equipment  programs.  The  plan  may  be  used  by  the  procuring 
activity  both  to  assess  and  approve  the  contractor's  approach 
and  methods  for  computer  program  (continued  •  see  page  2) 
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This  Data  Item  Description  applies  to  the  computer  resources 
portion  of  system  development  and  acquisition  and  can  be 
utilized  during  the  validation  and  subsequent  phases  of  the 
DOD  system  acquisition  cycle.  A  CPDP  may  be  obtained  pre- 
contractually  In  the  bidders'  proposals  and  may  be  a  product 
of  the  validation  contract  or  acquired  during  the  full  scale 
engineering  development  contract.  The  plan  may  be  maintained 
to  reflect  current  status  of  the  requirements.  The  CPDP  Is 
Intended  to  complement  other  contractual  management  plans 
which  address  disciplines  such  as  systems  engineering,  con¬ 
figuration  management,  and  test  and  evaluation.  Details  from 

P^hese  related  contractual  management  plans  or  other  dellver- 
ble  documents  may  be  Incorporated  by  reference,  but  the 
references  must  be  specific  enough  to  Indicate  precisely 
the  elements  of  the  plan.  Supersedes  DI-S-30567. 
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10.1  The  plan  shall  be  In  contractor  format  unless  a  specific  format  Is  specified  In 
the  Contract  Data  Requirements  List.  The  following  Items  shall  be  provided  as  a  minimum. 
Mhere  contractor  format  Is  used,  a  cross  reference  between  the  CPDP  and  the  following 
[items  will  be  Included. 


•.  Requirements  Assessment  Summary.  This  section  of  the  CPDP  shall  summ 
specifically  reference  documents  describing  the  contractor's  understanding  and 
nent  of  the  completeness  and  clarity  of: 


(1)  The  stated  computer  resource  requirements, 

(2)  The  definition  of  the  hardware  and  software  configuration  Items  and 
their  Interfaces, 

(3)  The  current  functional  allocation, 

(4)  Required  margins  end  budgets  for  critical  Items  such  as: 

(a)  Memory 

(b)  Execution  time 

(c)  Support  products, 

and  shall  Include: 


DD'’~!""1664 
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0I-S-30S67A  (continued) 

Preparation  Instructions  (continued) 


3.  Oescrlptlon/Purpose  (Continued) 

development  and  to  assist  In  monitoring  and  evaluating  the  contractor's 
efforts  during  development  and  test  of  the  products  defined  by  the  con¬ 
tract.  The  CROP  addresses,  summarizes  or  references  the  management  Issues 
In  a  single  document. 


10.  Preparation  Instruction  (Continued) 

(1)  A  suimnary  of  high  risk  and  uncertainty  Issues. 

(a)  Technical 

(b)  Cost 

(c)  Schedule 

(2)  Identification  of  proprietary  computer  resources  and  other 
used-as-1s  or  modifled-and-used  computer  resources  to  be 
used  during  the  contract  Including  subcontracted  resources. 

(3)  Approaches  with  backup  alternatives  to  minimize  or  over¬ 
come  the  risk  and  uncertainty  Issues. 

(4)  The  risk  Involving  long-lead  Items  and  their  status. 

(5)  Trade  and  optimization  studies  yet  to  be  conducted  and 
their  relationship  to  the  computer  resources. 

(6)  The  rationale  for  the  selection  of  the  computer  programming 
language(s)  If  not  specified  or  If  different  from  that 
specified. 

(7)  Identification  of  subcontract  Items  and  who  will  be  per¬ 
forming  the  subcontract  work. 

b.  Project  Objectives.  The  CPDP  shall  provide:  A  statement  of  the 
contractors  objectives  and  goals;  For  each  Computer  Program  Con¬ 
figuration  Item  (CPCI),  per  MIL-STO-483,  the  planned  allocation 
of  Its  functions  and  Interfaces;  For  each  CPCI,  budget  goals  for 
timing  memory.  Interface,  and  channel  capacity  (Including  redefini¬ 
tion,  when  required,  based  on  the  contractor's  approach)  end  mar¬ 
gins  for  growth  In  these  capacities  both  during  development  and 
future  operation;  The  relative  Importance  of  such  software  character¬ 
istics  as  reliability,  maintainability,  testability,  efficiency, 
portability,  Interoperability,  other  factors  from  technical  approach 
that  effect  the  CPDP,  etc. 
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01 -S- 30567 A  (continued) 

Preparation  Instructions  (continued) 


c.  Work  Definition.  The  CPDP  shell  provide:  The  work  tasks  required 

to  achieve  the  objectives  of  the  computer  pro grew  development  effort; 
Identification  of  the  necessary  development  steps  such  as  anelysls, 
design,  coding,  checkout.  Integration,  test,  acceptance  and  delivery 
for  each  CPCI  and  major  supporting  computer  resources  and  their  re¬ 
lationship  to  the  contractor's  Work  Breakdown  Structure  (UBS),  per 
NIl-STD-881,  Including  required  UBS  deviations;  Tasks  associated 
with  support  functions  (such  as  documentation,  configuration  manage¬ 
ment,  data  management,  management  reviews,  quality  assurance,  etc.) 
and  their  relationship  to  other  contractual  tasks;  Tasks  Involving 
Integration  and  test  of  CPCls,  Including  such  factors  as  separate 
test  configurations,  system  tests,  and  operational  tests  where 
applicable;  Major  development  steps  for  all  Identified  Computer 
Program  Components  (CPCs)  In  each  CPCI,  and  all  elements  of  the 
required  computer  resources  which  support  the  products. 

d.  Work  Schedule.  The  CPDP  shall  provide:  A  time  schedule  of  the  above 
work  elements,  based  on  the  contract  master  schedule.  Indicating 
Initiation,  Intermediate  (e.g.,  availability  of  draft  and  final  copies 
of  formal  and  Informal  documentation)  and  completion  times  for  all 
CPCls  and  time/performance  critical  Computer  Program  Components  (CPCs) 
Including  formal  and  Informal  milestones,  reviews,  audits,  key  meet¬ 
ings,  documentation  releases,  etc. 

e.  Activity  Network.  The  CPDP  shall  provide:  An  Activity  Network 
(e.g.,  PEtoTJ  compatible  with  the  Work  Schedule  of  computer  program 
development  efforts.  Including  Interface  activities  (such  as  hard¬ 
ware  development)  that  can  Impact  schedule  and  an  Identification 
of  critical  path  or  near  critical  path  elements;  To  the  extent  pos¬ 
sible,  an  identification  of  those  tasks  and  activity  paths  that 
might  be  most  strongly  Influenced  by  changes  In  program  requirements. 

f.  Organization.  The  CPDP  shall  provide:  A  delineation  of  the  con¬ 
tractor's  organizational  structures,  authorities,  responsibilities. 
Interfaces,  skill  requirements  and  lines  of  communications  necessary 
to  manage  and  execute  the  scheduled  activities  to  assure  proper  task 
completion;  The  relationships  assumed  between  the  contractor  and 
Independent  or  redundant  verification  and  validation  groups,  other 
contractors,  subcontractors,  the  procuring  agency,  the  support  agency, 
and  the  user  agency;  Identification  of  skill  requirements  and  key 
skilled  managers  and  employees  by  name. 

g.  Resource  Allocation.  The  CPDP  shall  provide:  An  allocation  of 
facilities,  laboratories,  computer  time,  test  equipments,  and  other 
relevant  resources  against  the  organizational  structures,  work 
elements,  schedules;  Identification  of  project-peculiar  resources 
required  such  as  special  purpose  hardware  and  computer  programs, 
govarnment-furnlshad  Items,  special  data,  etc.;  List  of  Items  which 
may  Impact  resources  such  as  high  risk  development  Items,  special 
security  requirements,  subcontractor  control,  etc.;  Presentation 
of  the  methods  to  be  used  to  assure  compatibility  of  the  procured 
systems  with  the  Intended  operational  physical  facilities. 


DI-S-30567A  (continued) 

Preparation  instruction  (continued) 


h.  Enqineerl no  Standards.  The  CPOP  shall  provide:  A  definition  or 
Identification  of  software  engineering  prect Ices  as  they  apply  to 
each  CPCI  and  computer  resource  support  product,  how  these  prac¬ 
tices  will  be  aialntalned,  and  how  their  use  will  be  assured;  Ex¬ 
amples  of  engineering  practices  Include*,  standards,  conventions, 
procedures,  and  rules  for  program  design,  structures;  display  and 
logic  standards;  Input/output  standards;  guidelines  for  program 
subdivision,  coding  techniques,  and  programing  languages;  data 
base  standards  and  other  disciplines  affecting  development. 

I.  Desl on  Assurance  Techniques.  The  CPOP  shall  provide:  A  defini¬ 
tion  of  the  techniques  used  for  design  analysis  end  control  to 
assure  completeness,  validity  traceability  to  requirements,  test¬ 
ability,  and  compliance  with  standards  and  practices;  The  approach 
to  preliminary  and  critical  design  reviews  such  as  Incremental 

or  multiple  reviews;  A  definition  of  the  design  evaluation  tech¬ 
niques  and  an  Identification  of  the  purpose,  application,  end 
validity  of  the  tools  used  such  as  computer  resource  support  pro¬ 
ducts,  simulations,  prototypes,  mathematical  models,  end  emulations; 
An  Identification  of  verification  and  validation  procedures  end 
aids  and  how  they  will  be  qualified  and  used  during  the  entire 
software  development  process;  A  statement  of  the  plan  fbr  defining, 
managing,  controlling,  and  reporting  the  status  of  budgets  on 
timing  and  memory,  and  technical  Interfaces  and  Interface  margins 
(such  as  I/O  channel  capacities  and  protocols).  Including  guide¬ 
lines  a^  checkpoints  for  Insuring  future  computer  program  growth 
capability,  modularity,  and  ease  of  modification. 

J.  Detailed  Design.  Coding,  and  Checkout.  The  CPOP  shall  provide:  A 
definition  or  the  procedures ,  steps  and  documents  associated  with 
detailed  design,  coding,  checkout,  review  and  acceptance  of  the 
Individual  modules  comprising  CPCs  and  CPCIs  Including  the  funtlons 
performed,  the  handling  of  the  Internal  and  external  Interfaces, 
and  the  control  of  parameters  like  memory  allocations,  operating 
sequences,  data  base  requirements  and  test  requirements;  Identifica¬ 
tion  of  computer  programs,  equipment  and  devices  required  to  support 
system  computer  hardware  and  to  facilitate  software  changes.  Includ¬ 
ing  diagnostic  software  for  all  computer  resources;  Criteria  and 
the  mechanism  for  acceptance  of  modules  for  Integration  and  test. 

k.  Pevelo went  Test  and  Evaluation.  The  CPOP  shall  provide:  A  pre¬ 
sentation  57  the  Integration  and  test  philosophy  and  approach  for 
CPCs  and  CPCIs,  how  this  philosophy  is  applied  In  the  design  and 
scheduling,  and  how  the  approach  leads  to  the  Preliminary  and  FPrmal 
Qualification  Tests;  A  definition  of  the  Interfaces  and  responsibil¬ 
ities  (such  as  training)  among  test  groups  and  other  project  partici¬ 
pants;  The  resource  ana  schedule  Impact  of  Independent  verification 
and  validation  activities. 
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Preparation  Instructions  (continued) 


l.  System  Test  and  evaluation.  The  CPDP  shall  provide:  Procedures  for 
the  coordination  and  support  of  CPC Is  and  other  computer  resources 
during  formal  system  testing  (and  during  Operational  Test  and 
Evaluation  as  applicable).  Including  necessary  training  and  trans- 
fer  of  data. 

m.  Anomaly  Control .  The  CPDP  shall  provide:  The  methods  or  system 
by  which  computer  software  anomalies  are  detected  and  documented 
during  all  test  and  evaluation,  corrections  are  provided,  and  con* 
figuration  control  Is  maintained.  Including  organization  respon¬ 
sibilities;  The  data  to  be  gathered  to  enable  assessment  of  the 
reliability  of  the  computer  resources,  reporting  periods,  data 
sources,  and  method  to  establishing  when  each  anomaly  (l.e.,  design, 
coding,  test,  etc.)  was  detected  and  corrected. 

n.  Management  Controls.  The  CPDP  shall  provide:  Relationships  between 
the  management  controls  of  the  CPDP  and  other  applicable  management 
plans  such  as:  the  Configuration  Management  Plan,  security  plan, 
and  the  Systems  Engineering  Management  Plan  (SEMP);  The  means  and 
criteria  for  management  assessment  and  control  of  the  development  pro¬ 
cess,  Including  subcontracts;  for  example,  mechanisms  for  Initiating 
management  actions  when  status  dictates  deviation  from  plan,  such  as 
resource  reallocation,  schedule  slippage,  cost  Increases,  or  per¬ 
formance  degradation. 

o.  Documentation.  The  CPDP  shall  provide:  Any  new  software  documenta- 
tion  tools  or  a  description  of  the  contractor  documentation  practice 
Including  techniques  Intended  for  use  on  the  subject  contract; 
Identification  of  the  specific  documentation  alternatives  and  the 
necessary  changes  to  the  content  and  format  of  the  specified  docu- 
mentation  requirements,  Including,  as  a  minimum,  a  description  of 
the  equivalency  features  between  the  specified  documentation  re¬ 
quirements  and  the  proposed  alternatlve(s);  A  description  of  the 
procedures  which  will  be  used  to  keep  the  Informal  documentation 
current;  A  description  of  the  approach  to  computer  program  In-line 
commenting. 

p.  Configuration  Management.  The  CPDP  shall  provide:  The  special 
aspects  of  computer  software  configuration  management  not  addressed 
In  the  overall  Configuration  Management  Plan  to  Include  the  or¬ 
ganizational  placement,  change,  deviation,  waiver  authorization 

and  control.  Internal  engineering  change  board,  configuration  Identi¬ 
fication,  computer  program  library  control,  the  control  or  changes  to 
any  non-dell vara ble  computer  programs,  the  handling  of  Interface 
control  between  the  hardware  and  the  software  of  the  system,  main¬ 
tenance  of  Independent  master  card/decks/tapes/discs,  control  of 
development  versions  of  CPCs  end  CPCIs,  handling  of  source  and  ob¬ 
ject  code,  control  over  test  environments,  relationship  to  program¬ 
ming  standards,  and  relationship  to  the  Quality  Assurance  function. 
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Preparation  Instructions  (continued) 

q.  Vendor/GFC  Coaputer  Resources .  The  CPDP  shell  provide:  The  pro* 
cedures  for  qualifying  and  documenting  the  present  adequacy  end 
the  adequacy  for  future  use  of  vendor- supplied  or  6FC  coaputer  re¬ 
sources  (hardware  and  software)  and  the  swans,  such  as  testlnq,  for 
accomodating  revision  of  vendor-supplied  coaputer  resources. 

r.  Support  Resources  for  the  Peploimnt  Phase.  The  CPDP  shall  provide 
The  recoamended  support  philosophy  and  the  resource  requlreaents 
for  use  after  the  full-scale  engineering  developaent  phase.  This 
section  of  the  CPDP  shall  suaaarlze  or  specifically  reference  the 
plan  for  the  transfer  of  coaputer  resources  Including  support  soft¬ 
ware  and  tools  to  the  appropriate  Air  Force  agencies. 


APPENDIX  E 


SOFTWARE  ACQUISITION  STANDARDS.  METHODOLOGIES, 

AND  TECHNIQUES 

Generally  speaking,  there  is  considerable  overlap  in  definition 
of  the  three  terms  -  standards,  methodologies,  and  techniques  -  which 
apply  to  software  acquisition.  For  purposes  of  discussion  within  this 
guidebook,  the  following  definitions  are  given: 

e  Standards  -  That  set  of  published  rules,  conventions, 
or  principles  accepted  as  generalized  guidance  in  soft¬ 
ware  acquisition. 

e  Methodologies  -  That  set  of  applied  concepts  used  in 
generating  design  and  code  for  software  acquisition. 

e  Techniques  -  That  set  of  procedural  implementation 
used  in  applying  software  acquisition  methodologies. 

STANDARDS 

Standards  of  time,  distance,  and  speed  are  commonly  used  every¬ 
day  without  the  user  being  fully  aware  of  the  fact.  The  importance  of 
standards  is  so  meaningful  that  a  United  States  Bureau  of  Standards 
exists  to  publish  and  safeguard  those  master  reference  devices.  Yet, 
when  the  term  "Programming  Standards,  "  for  example,  is  mentioned 
there  is  little  recognition  of  its  true  meaning.  When  properly  used, 
standards  simplify  an  individual  programmer's  job  by  eliminating  the 
need  for  tradeoff  studies  and  by  making  automated  tools  available. 

Standards  also  establish  a  uniform  basis  for  groups  of  programmers  to 
work  together  with  a  minimum  amount  of  confusion  and  misunderstanding. 
The  Air  Force  has  published  several  standards  in  an  attempt  to  establish 
a  common  basis  of  communication  and  understanding  in  software  acquisi¬ 
tion.  This  guidebook  does  not  attempt  to  address  each  of  them  for  they 
are  too  numerous,  however,  the  following  discussion  is  offered  to  encourage 
use  of  standards  to  a  practical  extent. 
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•  Programming  Practices  and  Standards 

Explicit  coding  practices  and  standards  are  very  useful  controls  for 
producing  software  programs  and  to  assure  proper  implementation.  The 
degree  of  usage  is  likely  to  be  proportional  to  the  degree  of  management 
emphasis  for  adherence  to  these  practices.  Adherence  to  standards  will 
make  the  software  far  more  intelligible  and  should  reduce  the  time  required 
for  debugging,  testing,  and  changing  the  software.  Documentation  is  more 
easily  relatable  to  "clean"  software  than  to  software  which  was  developed 
without  regard  to  standards.  Coding  practices,  standards,  and  their 
enforcement  can  be  used  as  schedule  status  factors.  Standards  can  be 
written  for  a  particular  program  as  well  as  those  more  universal  ones 
published  by  the  government.  The  following  list  of  available  standards  is 
offered  for  consideration  in  your  project 

e  Activity  Networks  -  The  requirement  that  a  chart  be 
drawn  showing  all  the  activities  to  be  completed  in  the 
project,  as  well  as  the  interactions  among  activities, 
provides  an  excellent  tool  for  ensuring  the  completeness 
of  a  project  plan  and  the  validity  of  the  schedule. 

e  Phase  Review  -  Formal  procedures  for  obtaining  manage¬ 
ment  and  technical  approval  at  the  end  of  a  task.  Avoid 
false  starts  and  permit  timely  decision  making. 

e  Configuration  Management  -  Constructing  a  baseline  speci¬ 
fication  and  establishing  a  control  procedure  to  authorize 
changes  to  the  baseline  are  necessary  in  the  control  of  a 
large  system. 

e  Machine  Room  Procedures  -  Better  service  can  be  obtained 
from  a  central  computer  facility  if  all  users  follow  standard 
procedures  for  job  submission,  priority  assignment,  and 
usage  forecasting. 

•  Top  Down  Programming  -  Direct  the  programmers  to 
design  their  program  by  first  defining  files  and  major 
component  structure  and  then  adding  calls  to  more 
detailed  subelements. 

e  Structured  Programming  -  By  following  valid  rules,  pro¬ 
grammers  tend  to  generate  software  that  is  easier  to 
read  and  test. 

e  Using  Library  Programs  -  Considerable  new  programming 
effort  can  be  avoided  by  multiple  programmer  use  of  exist¬ 
ing  programs. 
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•  Language* 


Not  all  phrases  are  as  explicitly  described  in  English  as  they  are  in 
Latin  (the  converse  is  true  for  other  phrases).  Similarly,  not  all  computer 
languages  have  die  same  degree  of  applicability  to  all  computer  tasks . 

The  selection  of  the  implementation  program  language(s)  should  be  viewed 
with  regard  to  all  phases  of  development  including  test  and  maintenance. 

The  applicability  of  a  computer  language  to  the  task  at  hand  plus  the  degree 
of  adherence  to  that  language  direcly  affect  programmer  progress  in 
generating  software.  Other  factors  that  influence  progress  are:  language 
version,  translator  availability,  compiler  stability,  efficiency  of  generated 
code,  and  compiler/assembler  code  apportionment. 

Higher  Order  Languages  (HOL)  are  generally  regarded  as  easier 
to  use,  however,  they  sometimes  occupy  more  room  in  the  computer 
memory  than  assembler  language.  Certain  programming  tasks  lend  them¬ 
selves  to  more  favorable  usage  of  HOL  than  other  tasks  do.  It  is  wise  to 
study  the  system  in-depth  and  decide  upon  the  type  of  language  to  use  prior 
to  any  contractual  agreements. 

METHODOLOGIES 

Methodology  is  a  term  that,  when  applied  to  software  development, 
attempts  to  describe  a  more  orderly  approach  to  generating  software 
in  modern  times  as  compared  to  earlier  software  efforts.  Sophistication 
of  software  programs  has  evolved  to  the  extent  it  is  no  longer  efficient 
(or  in  some  cases,  possible)  to  generate  software  with  an  unplanned 
(unstructured  ?  )  approach.  Methodologies  attempt  to  more  logically  design, 
code,  and  test  the  software.  Although  methodologies  are  not  a  panacea  for 
software  development,  they  do  offer  a  more  disciplined  approach  which 
results  in  improvements  to  program  quality,  manageability,  and  productiv¬ 
ity.  Descriptions  of  methodologies  are  sometimes  overlapping  and  vague. 
The  intent  of  this  section  is  to  acquaint  the  reader  with  certain  current 
concepts  of  the  "how  to's"  of  computer  program  development. 


% 
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•  Top  Down/ Bottom  Up  Design 

The  terms  top  down  and  bottom  up  have  been  widely  used  in  recent 
years  in  association  with  software  development.  Primarily  top  down 
begins  first  with  the  establishment  of  a  control  architecture  along  with 
interface  and  data  requirements.  Subsequently,  lower  level  functions  are 
designed,  and  the  process  continues  until  all  required  functions  and  sub¬ 
functions,  interfaces,  and  data  requirements  have  been  designed.  Top 
down  is  analogous  to  commencing  at  the  tip  of  an  Air  Force  organisational 
chart  and  working  downward  through  the  organizational  tiers.  Bottom 
up  is  simply  the  reverse  of  top  down  (start  at  the  lowest  organizational 
level  and  work  upward).  Software  designers  in  particular  have  used  the 
top  down  approach  and  through  successive  refinements  are  able  to  pro¬ 
duce  more  independent  "hunks"  of  software  (modules).  Both  top  down 
and  bottom  up  are  used  in  the  implementation  phase  (code,  integrate,  and 
test)  of  development;  however,  only  top  down  is  successful  in  the  design. 
One  word  of  caution  in  using  bottom  up  is  the  design  must  have  been 
completed  prior  to  commencing  botton  up  implementation  or  else  there 
is  a  risk  the  module  will  not  efficiently  apply  to  the  overall  design. 
Frequently,  coding  commences  prior  to  design  solidification,  therefore 
causing  considerable  recode. 

The  advantages  of  top  down  implementation  are:  integration  is 
continuous  and  fully  tested  prior  to  proceeding  to  the  next  level  down, 
software  quality  is  enhanced  through  earlier  detection  and  elimination  of 
design  problems  and  coding  errors,  and  the  need  for  writing  drivers  for 
module  testing  is  eliminated.  Few  advantages  are  attributed  to  bottom 
up  implementation  although  certain  specialized  programs,  such  as  the 
device  handlers  of  an  operating  system,  are  more  reasonably  accom¬ 
plished  by  this  approach. 

Figure  E-l  is  included  to  identify  design  techniques  which  are  con¬ 
sistently  recommended  plus  some  which  are  favored  under  certain  cir¬ 
cumstances. 


BASICS  OF  GOOD  DESIGN 


Figure  E-l*  Design  Considerations 


•  Modularity 

As  previously  mentioned  in  this  guidebook,  modularity  is  the  par¬ 
titioning  of  software  into  smaller  pieces  with  well-defined  inputs,  well- 
defined  processing,  and  well-defined  outputs.  The  closer  to  an  indepen¬ 
dent  "box"  these  smaller  pieces  become  the  more  modular  the  system. 
Modularization  provides  a  systematic  method  for  isolating  problems  and 
minimizes  the  effects  of  a  module  change  upon  other  modules.  A  pre¬ 
dictable  module  is  one  that  operates  identically  the  same  each  time  the 
same  input  is  supplied.  The  advantage  of  predictability  to  management 
visibility  is  obvious.  In  summary,  modularity  breaks  complexity  into 
a  group  of  simpler  pieces  and  enhances  development  and  management 
visibility. 

•  Structured  Design 

Structured  design  combines  both  top  down  design  and  modularity. 

Its  main  emphasis  is  to  break  functions  into  modules  such  that  the  input, 
processing,  and  output  of  each  module  approximate  a  functional  require¬ 
ment.  Those  programs  whose  functional  requirements  are  easily  and 
clearly  defined  are  most  promising  for  using  this  approach. 

TECHNIQUES 

Techniques  associated  with  software  development  are  generally 
conceived  to  b£  of  more  detail  than  methodologies.  An  example  to 
illustrate  is;  all  pilots  use  the  method  of  rudder  and  stick  controls  to  , 
approach  the  runway  in  crosswind  conditions,  yet  some  pilots  use  the 
technique  of  cross  controlling  while  others  use  the  technique  of  crabbing. 
Simply  stated,  a  technique  is  the  procedure  by  which  a  methodology  is 
applied  or  implemented.  . 

•  Verification  and  Validation 

It  is  possible  to  apply  the  technique  of  Verification  and  Validation 
(V&V)  through  use  of  an  independent  contractor  or  through  the  developing 
contractor.  In  either  case,  the  process  of  V&V  should  be  initiated  long 
before  the  completion  of  the  first  pass  through  the  development  to  take 
advantage  of  early  identification  of  problems  and  their  corrections.  V&V 
is  an  attempt  at  determining  the  validity  of  computer  programs  but  a 
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complete  check  of  all  variables  and  combinations  is  expensive  and  time 
consuming.  Usually  V&V  encompasses  software  exercises,  code  checks, 
functional  checks,  requirements  traceability,  and  other  analytical  con¬ 
siderations  which  give  an  indication  of  program  quality.  The  process  is 
most  efficiently  applied  through  use  of  a  group  that  is  independent  from 
the  design/code  groups  because  it  forces  incremental  proof  of  the  validity 
c* f  the  developed  programs. 

•  Organic  and  Inorganic  Development 

AFM  26-1  requires  a  study  to  be  conducted  that  determines 
whether  organic  development  will  be  accomplished  or  not.  Subsequent 
to  the  determination,  the  actual  task  is  undertaken.  Most  tasks  are 
desirable  for  organic  (in-house)  development  because  of  the  residual 
expertise  and  experience  left  within  the  Air  Force  subsequent  to  the 
development.  The  demand  for  expertise  levels  and  personnel  resources 
sometimes  exceed  those  resources  available  and  thus  the  efforts  are 
contracted  to  industry.  If  the  tasks  are  contracted,  then  a  mechanism  for 
transferring  needed  expertise  and  system  knowledge  from  the  contractor  to 
the  Air  Force  must  be  identified  and  enforced.  Quite  often,  a  complex 
system  is  developed  and  the  contractor  and/or  the  Air  Force  do  not  make 
adequate  allowance  for  knowledge  transfer  thus  endangering  long  term 
support  and  operation  of  the  system. 

*  Other  Techniques 

Sometimes  it  is  desirable  to  break  the  development  into  more  than 
one  contract.  This  assigns  the  integration  role  to  the  Air  Force  or  an 
integrating  contractor  as  opposed  to  a  prime  contractor.  Integration 
requires  technical  competence  and  should  only  be  undertaken  organically 
if  the  computer  programs  are  almost  independent  of  each  other  and  a 
very  low  system  risk  is  envisioned. 

Seldom  is  it  desirable  to  break  the  software  away  entirely  from  the 
hardware.  Systems  are  growing  more  and  more  complex  and  the  soft¬ 
ware  plays  such  an  integral  role  in  the  overall  system  that  breaking 
hardware  and  software  apart  is  impractical.  The  current  tendency  is 
toward  "systems"  considerations  instead  of  "hardware"  and  "software" 
for  development  and  support. 
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